{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SQL-Agent\n",
    "\n",
    "- Author: [Jinu Cho](https://github.com/jinucho)\n",
    "- Peer Review: \n",
    "- Proofread : [Chaeyoon Kim](https://github.com/chaeyoonyunakim)\n",
    "- This is a part of [LangChain Open Tutorial](https://github.com/LangChain-OpenTutorial/LangChain-OpenTutorial)\n",
    "\n",
    "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/LangChain-OpenTutorial/LangChain-OpenTutorial/blob/main/17-LangGraph/03-Use-Cases/09-LangGraph-SQL-Agent.ipynb)[![Open in GitHub](https://img.shields.io/badge/Open%20in%20GitHub-181717?style=flat-square&logo=github&logoColor=white)](https://github.com/LangChain-OpenTutorial/LangChain-OpenTutorial/blob/main/17-LangGraph/03-Use-Cases/09-LangGraph-SQL-Agent.ipynb)\n",
    "## Overview\n",
    "\n",
    "In this tutorial, we will build an agent step-by-step that can answer questions\n",
    "\n",
    "![sql-agent-work-flow](./assets/09-sql-agent-work-flow.png)\n",
    "\n",
    "**Key Topics** :\n",
    "1. **Database** : Set up an SQLite database and load the Chinook sample database.\n",
    "2. **Utility Functions** : Define utility functions for implementing the agent.\n",
    "3. **Tool Definitions** : Define tools for interacting with the database.\n",
    "4. **Workflow Definition** : Define the workflow (graph) for the agent.\n",
    "5. **Graph Visualization** : Visualize the defined workflow graph.\n",
    "6. **Agent Execution** : Execute the agent and review the results.\n",
    "7. **Evaluation** : Evaluate the agent and compare its performance.\n",
    "\n",
    "### Table of Contents\n",
    "\n",
    "- [Overview](#overview)\n",
    "- [Environment Setup](#environment-setup)\n",
    "- [Agent for Interacting with SQL Databases](#agent-for-interacting-with-sql-databases)\n",
    "- [Set Up the Database](#set-up-the-database)\n",
    "- [Utility Functions](#utility-functions)\n",
    "- [SQL Query Execution Tools](#sql-query-execution-tools)\n",
    "- [SQLDatabaseToolkit Tools](#sqldatabasetoolkit-tools)\n",
    "- [SQL Query Checker](#sql-query-checker)\n",
    "- [Graph Definition](#graph-definition)\n",
    "- [Evaluating an SQL Agent Using LangSmith Evaluator](#evaluating-an-sql-agent-using-langsmith-evaluator)\n",
    "\n",
    "### References\n",
    "- [SQLite Sample Database](https://www.sqlitetutorial.net/sqlite-sample-database/)\n",
    "- [Langchain core](https://python.langchain.com/api_reference/core/messages.html#module-langchain_core)\n",
    "- [SQLDatabaseToolkit](https://python.langchain.com/api_reference/community/agent_toolkits/langchain_community.agent_toolkits.sql.toolkit.SQLDatabaseToolkit.html#langchain_community.agent_toolkits.sql.toolkit.SQLDatabaseToolkit)\n",
    "- [LangGraph](https://langchain-ai.github.io/langgraph)\n",
    "- [LangSmith.Client](https://docs.smith.langchain.com/reference/python/client/langsmith.client.Client)\n",
    "- [LangSmith.evaluation](https://docs.smith.langchain.com/reference/python/evaluation)\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Environment Setup\n",
    "\n",
    "Setting up your environment is the first step. See the [Environment Setup](https://wikidocs.net/257836) guide for more details.\n",
    "\n",
    "\n",
    "**[Note]**\n",
    "\n",
    "The langchain-opentutorial is a package of easy-to-use environment setup guidance, useful functions and utilities for tutorials.\n",
    "\n",
    "Check out the  [```langchain-opentutorial```](https://github.com/LangChain-OpenTutorial/langchain-opentutorial-pypi) for more details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture --no-stderr\n",
    "%pip install langchain-opentutorial"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m24.3.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m25.0\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "# Install required packages\n",
    "from langchain_opentutorial import package\n",
    "\n",
    "package.install(\n",
    "    [\n",
    "        \"langsmith\",\n",
    "        \"langchain\",\n",
    "        \"langgraph\",\n",
    "        \"langchain_openai\",\n",
    "        \"langchain_community\",\n",
    "    ],\n",
    "    verbose=False,\n",
    "    upgrade=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can alternatively set ```OPENAI_API_KEY``` in ```.env``` file and load it. \n",
    "\n",
    "[Note] This is not necessary if you've already set ```OPENAI_API_KEY``` in previous steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Environment variables have been set successfully.\n"
     ]
    }
   ],
   "source": [
    "# Set environment variables\n",
    "from langchain_opentutorial import set_env\n",
    "\n",
    "set_env(\n",
    "    {\n",
    "        \"OPENAI_API_KEY\": \"\",\n",
    "        \"LANGCHAIN_API_KEY\": \"\",\n",
    "        \"LANGCHAIN_TRACING_V2\": \"true\",\n",
    "        \"LANGCHAIN_ENDPOINT\": \"https://api.smith.langchain.com\",\n",
    "        \"LANGCHAIN_PROJECT\": \"09-SQL-Agent\",\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load environment variables\n",
    "# Reload any variables that need to be overwritten from the previous cell\n",
    "\n",
    "from dotenv import load_dotenv\n",
    "\n",
    "load_dotenv(override=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set LLM Model\n",
    "MODEL_NAME = \"gpt-4o-mini\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Agent for Interacting with SQL Databases\n",
    "\n",
    "\n",
    "The workflow of an agent that executes SQL queries is as follows:\n",
    "\n",
    "1. **Understanding the Database Schema** : Retrieve a list of available tables.\n",
    "2. **Selecting Relevant Tables** : Identify tables related to the question.\n",
    "3. **Fetching DDL** : Obtain the schema definitions (DDL) of the selected tables.\n",
    "4. **Generating Queries** : Construct SQL queries based on the question and DDL information.\n",
    "5. **Query Validation** : Use an LLM to review for common errors and refine the query.\n",
    "6. **Query Execution and Error Handling** : Execute the query on the database engine and handle errors to ensure successful execution.\n",
    "7. **Generating Responses** : Provide the final answer based on the query results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set Up the Database\n",
    "\n",
    "In this tutorial, we will create an SQLite database. SQLite is a lightweight database that is easy to set up and use.\n",
    "\n",
    "For this tutorial, we will load the Chinook sample database, which represents a digital media store.\n",
    "\n",
    "You can find more information about the Chinook database [here](https://www.sqlitetutorial.net/sqlite-sample-database/).\n",
    "\n",
    "First, download the Chinook database to use for the exercises in this tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "File already exists\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "import os\n",
    "\n",
    "if os.path.exists(\"data/Chinook.db\"):\n",
    "    print(\"File already exists\")\n",
    "else:\n",
    "    print(\"Downloading file...\")\n",
    "    url = \"https://storage.googleapis.com/benchmarks-artifacts/chinook/Chinook.db\"\n",
    "    response = requests.get(url)\n",
    "    if response.status_code == 200:\n",
    "        os.makedirs(\"data\", exist_ok=True)\n",
    "        with open(\"data/Chinook.db\", \"wb\") as file:\n",
    "            file.write(response.content)\n",
    "        print(\"File downloaded and saved as Chinook.db\")\n",
    "    else:\n",
    "        print(f\"Failed to download the file. Status code: {response.status_code}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using the Chinook Database\n",
    "\n",
    "Next, we will create an ```SQLDatabase``` tool using the downloaded Chinook database and execute a sample query, **\"SELECT * FROM Artist LIMIT 5;\"** ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "db dialect : sqlite\n",
      "table names in db : ['Album', 'Artist', 'Customer', 'Employee', 'Genre', 'Invoice', 'InvoiceLine', 'MediaType', 'Playlist', 'PlaylistTrack', 'Track']\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\"[(1, 'AC/DC'), (2, 'Accept'), (3, 'Aerosmith'), (4, 'Alanis Morissette'), (5, 'Alice In Chains')]\""
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_community.utilities import SQLDatabase\n",
    "\n",
    "# Create an SQLDatabase instance from the SQLite database file\n",
    "db = SQLDatabase.from_uri(\"sqlite:///data/Chinook.db\")\n",
    "\n",
    "# Print the database dialect (sqlite)\n",
    "print(f\"db dialect : {db.dialect}\")\n",
    "\n",
    "# Print the list of usable table names in the database\n",
    "print(f\"table names in db : {db.get_usable_table_names()}\")\n",
    "\n",
    "# Execute an SQL query\n",
    "db.run(\"SELECT * FROM Artist LIMIT 5;\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Utility Functions\n",
    "\n",
    "Let's define several utility functions to assist in implementing the agent.\n",
    "\n",
    "In particular, wrap ```ToolNode``` to include error handling and the capability to relay errors back to the agent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Any\n",
    "\n",
    "from langchain_core.messages import ToolMessage\n",
    "from langchain_core.runnables import RunnableLambda, RunnableWithFallbacks\n",
    "from langgraph.prebuilt import ToolNode\n",
    "\n",
    "\n",
    "# Error handling function\n",
    "def handle_tool_error(state) -> dict:\n",
    "    # Check error information\n",
    "    error = state.get(\"error\")\n",
    "    # Check tool information\n",
    "    tool_calls = state[\"messages\"][-1].tool_calls\n",
    "    # Wrap with ToolMessage and return\n",
    "    return {\n",
    "        \"messages\": [\n",
    "            ToolMessage(\n",
    "                content=f\"Here is the error: {repr(error)}\\n\\nPlease fix your mistakes.\",\n",
    "                tool_call_id=tc[\"id\"],\n",
    "            )\n",
    "            for tc in tool_calls\n",
    "        ]\n",
    "    }\n",
    "\n",
    "\n",
    "# Create a ToolNode to handle errors and surface them to the agent\n",
    "def create_tool_node_with_fallback(tools: list) -> RunnableWithFallbacks[Any, dict]:\n",
    "    \"\"\"\n",
    "    Create a ToolNode with a fallback to handle errors and surface them to the agent.\n",
    "    \"\"\"\n",
    "    # Add fallback behavior for error handling to the ToolNode\n",
    "    return ToolNode(tools).with_fallbacks(\n",
    "        [RunnableLambda(handle_tool_error)], exception_key=\"error\"\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SQL Query Execution Tools\n",
    "\n",
    "To enable the agent to interact with the database, we define several tools.\n",
    "\n",
    "### Defined Tools:\n",
    "- ```list_tables_tool``` : Retrieves a list of available tables in the database.\n",
    "- ```get_schema_tool```  : Fetches the DDL (Data Definition Language) of a table.\n",
    "- ```db_query_tool```  : Executes queries and retrieves results or returns error messages if the query fails.\n",
    "\n",
    "### Note:\n",
    "DDL (Data Definition Language) refers to SQL commands used to define or modify the structure and schema of a database. It includes commands to create, alter, and delete database objects like tables, indexes, and views.\n",
    "\n",
    "### Key DDL Commands:\n",
    "- **CREATE** : Creates database objects.\n",
    "  - Example: CREATE TABLE users (id INT, name VARCHAR(100));\n",
    "- **ALTER** : Modifies existing database objects.\n",
    "  - Example: ALTER TABLE users ADD COLUMN email VARCHAR(100);\n",
    "- **DROP** : Deletes database objects.\n",
    "  - Example: DROP TABLE users;\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SQLDatabaseToolkit Tools\n",
    "\n",
    "The following tools are provided to interact with the SQL database:\n",
    "\n",
    "1. ```QuerySQLDatabaseTool```\n",
    "   - **Function** : Executes SQL queries and returns results.\n",
    "   - **Input** : Valid SQL query.\n",
    "   - **Output** : Database results or error messages.\n",
    "   - **Error Handling** :\n",
    "     - Rewrites and retries the query if errors occur.\n",
    "     - Resolves \"unknown column\" errors by checking table fields using ```sql_db_schema``` .\n",
    "\n",
    "2. ```InfoSQLDatabaseTool```\n",
    "   - **Function** : Retrieves table schemas and sample data.\n",
    "   - **Input** : Comma-separated list of table names.\n",
    "   - **Usage Example** : table1, table2, table3\n",
    "   - **Note** : Check table existence using ```sql_db_list_tables``` before using this tool.\n",
    "\n",
    "3. ```ListSQLDatabaseTool```\n",
    "   - **Function** : Retrieves a list of tables in the database.\n",
    "\n",
    "4. ```QuerySQLCheckerTool```\n",
    "   - **Function** : Validates SQL queries before execution.\n",
    "   - **Validation Checks** :\n",
    "     - Use of **NULL** values and **NOT IN** .\n",
    "     - Suitability of **UNION** vs **UNION ALL** .\n",
    "     - Correct range settings for **BETWEEN** .\n",
    "     - Data type consistency.\n",
    "     - Proper quoting of identifiers.\n",
    "     - Correct number of function arguments.\n",
    "     - Proper type casting.\n",
    "     - Accuracy of join columns.\n",
    "   - **Features** : Uses a GPT-4o model for validation.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "class_name : <class 'langchain_community.tools.sql_database.tool.QuerySQLDatabaseTool'>\n",
      "description : Input to this tool is a detailed and correct SQL query, output is a result from the database. If the query is not correct, an error message will be returned. If an error is returned, rewrite the query, check the query, and try again. If you encounter an issue with Unknown column 'xxxx' in 'field list', use sql_db_schema to query the correct table fields.\n",
      "\n",
      "class_name : <class 'langchain_community.tools.sql_database.tool.InfoSQLDatabaseTool'>\n",
      "description : Input to this tool is a comma-separated list of tables, output is the schema and sample rows for those tables. Be sure that the tables actually exist by calling sql_db_list_tables first! Example Input: table1, table2, table3\n",
      "\n",
      "class_name : <class 'langchain_community.tools.sql_database.tool.ListSQLDatabaseTool'>\n",
      "description : Input is an empty string, output is a comma-separated list of tables in the database.\n",
      "\n",
      "class_name : <class 'langchain_community.tools.sql_database.tool.QuerySQLCheckerTool'>\n",
      "description : Use this tool to double check if your query is correct before executing it. Always use this tool before executing a query with sql_db_query!\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from langchain_community.agent_toolkits.sql.toolkit import SQLDatabaseToolkit\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "# Create SQLDatabaseToolkit\n",
    "toolkit = SQLDatabaseToolkit(db=db, llm=ChatOpenAI(model=MODEL_NAME))\n",
    "\n",
    "# Get the list of available tools from the SQLDatabaseToolkit\n",
    "tools = toolkit.get_tools()\n",
    "for tool in tools:\n",
    "    print(f\"class_name : {tool.__class__}\")\n",
    "    print(f\"description : {tool.description}\")\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Below is an example of executing ```list_tables_tool``` and ```get_schema_tool``` ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "list_tables :\n",
      "`Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track\n",
      "----------------------------------------------------------------------------------------------------\n",
      "schema :\n",
      "\n",
      "CREATE TABLE \"Artist\" (\n",
      "\t\"ArtistId\" INTEGER NOT NULL, \n",
      "\t\"Name\" NVARCHAR(120), \n",
      "\tPRIMARY KEY (\"ArtistId\")\n",
      ")\n",
      "\n",
      "/*\n",
      "3 rows from Artist table:\n",
      "ArtistId\tName\n",
      "1\tAC/DC\n",
      "2\tAccept\n",
      "3\tAerosmith\n",
      "*/\n"
     ]
    }
   ],
   "source": [
    "# Select the tool for listing available tables in the database\n",
    "list_tables_tool = next(tool for tool in tools if tool.name == \"sql_db_list_tables\")\n",
    "\n",
    "# Select the tool for retrieving the DDL of a specific table\n",
    "get_schema_tool = next(tool for tool in tools if tool.name == \"sql_db_schema\")\n",
    "\n",
    "# Print the list of all tables in the database\n",
    "print(f'list_tables :\\n`{list_tables_tool.invoke(\"\")}')\n",
    "\n",
    "print(\"-\" * 100)\n",
    "\n",
    "# Print the DDL information for the Artist table\n",
    "print(f'schema :\\n{get_schema_tool.invoke(\"Artist\")}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, define ```db_query_tool``` .\n",
    "\n",
    "In ```db_query_tool``` case, it executes queries against the database and returns the results.\n",
    "\n",
    "If an error occurs, it returns an error message."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.tools import tool\n",
    "\n",
    "\n",
    "# Query execution tool\n",
    "@tool\n",
    "def db_query_tool(query: str) -> str:\n",
    "    \"\"\"\n",
    "    Run SQL queries against a database and return results\n",
    "    Returns an error message if the query is incorrect\n",
    "    If an error is returned, rewrite the query, check, and retry\n",
    "    \"\"\"\n",
    "    # Execute query\n",
    "    result = db.run_no_throw(query)\n",
    "\n",
    "    # Error: Return error message if no result\n",
    "    if not result:\n",
    "        return \"Error: Query failed. Please rewrite your query and try again.\"\n",
    "    # Success: Return the query execution result\n",
    "    return result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When executed successfully."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[(1, 'AC/DC'), (2, 'Accept'), (3, 'Aerosmith'), (4, 'Alanis Morissette'), (5, 'Alice In Chains'), (6, 'Antônio Carlos Jobim'), (7, 'Apocalyptica'), (8, 'Audioslave'), (9, 'BackBeat'), (10, 'Billy Cobham')]\n"
     ]
    }
   ],
   "source": [
    "# Select the top 10 rows from the Artist table and print the execution result\n",
    "print(db_query_tool.invoke(\"SELECT * FROM Artist LIMIT 10;\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When an error occurs.\n",
    "\n",
    "This will produce an error due to the incorrect SQL syntax (\"LIMITS\" instead of \"LIMIT\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error: (sqlite3.OperationalError) near \"10\": syntax error\n",
      "[SQL: SELECT * FROM Artist LIMITS 10;]\n",
      "(Background on this error at: https://sqlalche.me/e/20/e3q8)\n"
     ]
    }
   ],
   "source": [
    "# Attempt to select the top 10 rows from the Artist table and print the result\n",
    "print(db_query_tool.invoke(\"SELECT * FROM Artist LIMITS 10;\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SQL Query Checker\n",
    "\n",
    "Next, we will use an LLM to check for common mistakes in SQL queries.\n",
    "\n",
    "Strictly speaking, this is not a tool, but it will be added as a node in the workflow later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "# Define a system message to check SQL queries for common mistakes\n",
    "query_check_system = \"\"\"You are a SQL expert with a strong attention to detail.\n",
    "Double check the SQLite query for common mistakes, including:\n",
    "- Using NOT IN with NULL values\n",
    "- Using UNION when UNION ALL should have been used\n",
    "- Using BETWEEN for exclusive ranges\n",
    "- Data type mismatch in predicates\n",
    "- Properly quoting identifiers\n",
    "- Using the correct number of arguments for functions\n",
    "- Casting to the correct data type\n",
    "- Using the proper columns for joins\n",
    "\n",
    "If there are any of the above mistakes, rewrite the query. If there are no mistakes, just reproduce the original query.\n",
    "\n",
    "You will call the appropriate tool to execute the query after running this check.\"\"\"\n",
    "\n",
    "# Create the prompt\n",
    "query_check_prompt = ChatPromptTemplate.from_messages(\n",
    "    [(\"system\", query_check_system), (\"placeholder\", \"{messages}\")]\n",
    ")\n",
    "\n",
    "# Create the Query Checker chain\n",
    "query_check = query_check_prompt | ChatOpenAI(\n",
    "    model=MODEL_NAME, temperature=0\n",
    ").bind_tools([db_query_tool], tool_choice=\"db_query_tool\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Verify the Fix for Incorrect Query\n",
    "\n",
    "An incorrect query was executed to check if the issue is properly handled and fixed.\n",
    "\n",
    "**Remark** :  \n",
    "The query used ```LIMITS``` instead of ```LIMIT``` .\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'name': 'db_query_tool', 'args': {'query': 'SELECT * FROM Artist LIMIT 10;'}, 'id': 'call_oLpZXFYZmlUJ8sMbqthr4B1U', 'type': 'tool_call'}\n"
     ]
    }
   ],
   "source": [
    "# Execute the query check node using the user's message\n",
    "response = query_check.invoke(\n",
    "    {\"messages\": [(\"user\", \"SELECT * FROM Artist LIMITS 10;\")]}\n",
    ")\n",
    "print(response.tool_calls[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The input query contained an error, but the generated query was successfully corrected.\n",
    "\n",
    "**Correction**: LIMITS → LIMIT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Graph Definition\n",
    "\n",
    "Define the workflow for the agent.\n",
    "\n",
    "The agent first forcefully invokes the ```list_tables_tool``` to retrieve the available tables from the database. After this, it follows the steps mentioned earlier in the tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Annotated, Literal\n",
    "\n",
    "from langchain_core.messages import AIMessage\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "from pydantic import BaseModel, Field\n",
    "from typing_extensions import TypedDict\n",
    "\n",
    "from langgraph.graph import END, StateGraph, START\n",
    "from langgraph.graph.message import AnyMessage, add_messages\n",
    "from langgraph.checkpoint.memory import MemorySaver\n",
    "\n",
    "\n",
    "# Define the agent's state\n",
    "class State(TypedDict):\n",
    "    messages: Annotated[list[AnyMessage], add_messages]\n",
    "\n",
    "\n",
    "# Create a new graph\n",
    "workflow = StateGraph(State)\n",
    "\n",
    "\n",
    "# Add a node for the first tool call\n",
    "def first_tool_call(state: State) -> dict[str, list[AIMessage]]:\n",
    "    return {\n",
    "        \"messages\": [\n",
    "            AIMessage(\n",
    "                content=\"\",\n",
    "                tool_calls=[\n",
    "                    {\n",
    "                        \"name\": \"sql_db_list_tables\",\n",
    "                        \"args\": {},\n",
    "                        \"id\": \"initial_tool_call_abc123\",\n",
    "                    }\n",
    "                ],\n",
    "            )\n",
    "        ]\n",
    "    }\n",
    "\n",
    "\n",
    "# Define a function to check query accuracy with a model\n",
    "def model_check_query(state: State) -> dict[str, list[AIMessage]]:\n",
    "    \"\"\"\n",
    "    Use this tool to check that your query is correct before you run it\n",
    "    \"\"\"\n",
    "    return {\"messages\": [query_check.invoke({\"messages\": [state[\"messages\"][-1]]})]}\n",
    "\n",
    "\n",
    "# Add a node for the first tool call\n",
    "workflow.add_node(\"first_tool_call\", first_tool_call)\n",
    "\n",
    "# Add nodes for the first two tools\n",
    "workflow.add_node(\n",
    "    \"list_tables_tool\", create_tool_node_with_fallback([list_tables_tool])\n",
    ")\n",
    "workflow.add_node(\"get_schema_tool\", create_tool_node_with_fallback([get_schema_tool]))\n",
    "\n",
    "# Add a model node to select relevant tables based on the question and available tables\n",
    "model_get_schema = ChatOpenAI(model=MODEL_NAME, temperature=0).bind_tools(\n",
    "    [get_schema_tool]\n",
    ")\n",
    "workflow.add_node(\n",
    "    \"model_get_schema\",\n",
    "    lambda state: {\n",
    "        \"messages\": [model_get_schema.invoke(state[\"messages\"])],\n",
    "    },\n",
    ")\n",
    "\n",
    "# Add a model node to generate a query based on the question and schema\n",
    "QUERY_GEN_INSTRUCTION = \"\"\"You are a SQL expert with a strong attention to detail.\n",
    "\n",
    "You can define SQL queries, analyze queries results and interpretate query results to response an answer.\n",
    "\n",
    "Read the messages bellow and identify the user question, table schemas, query statement and query result, or error if they exist.\n",
    "\n",
    "1. If there's not any query result that make sense to answer the question, create a syntactically correct SQLite query to answer the user question. DO NOT make any DML statements (INSERT, UPDATE, DELETE, DROP etc.) to the database.\n",
    "\n",
    "2. If you create a query, response ONLY the query statement. For example, \"SELECT id, name FROM pets;\"\n",
    "\n",
    "3. If a query was already executed, but there was an error. Response with the same error message you found. For example: \"Error: Pets table doesn't exist\"\n",
    "\n",
    "4. If a query was already executed successfully interpretate the response and answer the question following this pattern: Answer: <<question answer>>. \n",
    "    For example: \"Answer: There three cats registered as adopted\"\n",
    "\"\"\"\n",
    "\n",
    "query_gen_prompt = ChatPromptTemplate.from_messages(\n",
    "    [(\"system\", QUERY_GEN_INSTRUCTION), (\"placeholder\", \"{messages}\")]\n",
    ")\n",
    "\n",
    "\n",
    "query_gen = query_gen_prompt | ChatOpenAI(model=MODEL_NAME, temperature=0).bind_tools(\n",
    "    [model_check_query]\n",
    ")\n",
    "\n",
    "\n",
    "# Define conditional edges\n",
    "def should_continue(state: State) -> Literal[END, \"correct_query\", \"query_gen\"]:\n",
    "    message = state[\"messages\"][-1].content\n",
    "\n",
    "    # 1) Terminate if the message starts with \"Answer:\"\n",
    "    if message.startswith(\"Answer:\"):\n",
    "        return END\n",
    "\n",
    "    # 2) Follow existing logic for other cases\n",
    "    elif message.startswith(\"Error:\"):\n",
    "        return \"query_gen\"\n",
    "    else:\n",
    "        return \"correct_query\"\n",
    "\n",
    "\n",
    "# Define the query generation node\n",
    "def query_gen_node(state: State):\n",
    "    message = query_gen.invoke(state)\n",
    "\n",
    "    # If the LLM makes incorrect tool calls, return error messages\n",
    "    tool_messages = []\n",
    "    message.pretty_print()\n",
    "    if message.tool_calls:\n",
    "        for tc in message.tool_calls:\n",
    "            tool_messages.append(\n",
    "                ToolMessage(\n",
    "                    content=f\"Error: The wrong tool was called: {tc['name']}. Please fix your mistakes. Remember to only call SubmitFinalAnswer to submit the final answer. Generated queries should be outputted WITHOUT a tool call.\",\n",
    "                    tool_call_id=tc[\"id\"],\n",
    "                )\n",
    "            )\n",
    "    else:\n",
    "        tool_messages = []\n",
    "    return {\"messages\": [message] + tool_messages}\n",
    "\n",
    "\n",
    "# Add the query generation node\n",
    "workflow.add_node(\"query_gen\", query_gen_node)\n",
    "\n",
    "# Add a node to check the query with the model before execution\n",
    "workflow.add_node(\"correct_query\", model_check_query)\n",
    "\n",
    "# Add a node to execute the query\n",
    "workflow.add_node(\"execute_query\", create_tool_node_with_fallback([db_query_tool]))\n",
    "\n",
    "# Specify edges between nodes\n",
    "workflow.add_edge(START, \"first_tool_call\")\n",
    "workflow.add_edge(\"first_tool_call\", \"list_tables_tool\")\n",
    "workflow.add_edge(\"list_tables_tool\", \"model_get_schema\")\n",
    "workflow.add_edge(\"model_get_schema\", \"get_schema_tool\")\n",
    "workflow.add_edge(\"get_schema_tool\", \"query_gen\")\n",
    "workflow.add_conditional_edges(\n",
    "    \"query_gen\",\n",
    "    should_continue,\n",
    ")\n",
    "workflow.add_edge(\"correct_query\", \"execute_query\")\n",
    "workflow.add_edge(\"execute_query\", \"query_gen\")\n",
    "\n",
    "# Compile the workflow into an executable app\n",
    "app = workflow.compile(checkpointer=MemorySaver())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaIAAALoCAIAAAC1QmavAAAAAXNSR0IArs4c6QAAIABJREFUeJzs3Wd8FOXaBvB7e03vPYGEhECA0IsB6UWQIr0IKCAqKqJ4ELAg9oICIiIiSBEpEZDepbdQAySE9L7pZbOb7e+H9Y2IEJIhyWaG6/87HzZTnufekb3OM51nsVgIAIC7+LYuAACgfiHmAIDjEHMAwHGIOQDgOMQcAHAcYg4AOE7w4Ycf2roGAI64XV50vlilt5gOqNKztOpmSsd8vXZ7VlJj/pyr0wQrHHJ1mqSKUkexRMjj4NCHg18JoCElVZSuSI49lJdRbjRcLy3I02nUBqPBbNKZTYX6yhKDvpF/rjSZivSV+TrtPlXaN3evGizmdG15qUFv6+1al3i4PBiAmfjy4iCF/Y7sZGexNNLRlU88W1dUN7IqKzZl3BniGfSUi5eta6kbiDmAWjMTfX4nJtzeOcrF29a11JdUTXmY0vFaaUE39ocdYg6gdkqN+nKDPkOrbqZ0tHUt9W5vbqrBYn4xINzWhTwWxBxALWRp1bk6baDcztaFNJyrpQU9XL1FbD41weLSARrYHXXJDymxT1TGEVGkg2uapvx4fpatC2EOozmAmtKYjFqT0dZV2MaZopx8nZale6+IOYAa2Z6d2MHRQykU2boQm1GbDK4iqYKFWwA7rQCPFp2drDWZnuSMIyKlQJSt09i6CiYwmgN4BDNRbGmBr0xp60Js74AqTcgXjPEJtnUhtYPRHMAj8Ij8GjbjVLlZ8XE3HqeFuNvX81TZdVfR3/q4+2doyuu82fqGmAN4hJeuHlfptA3W3Z342GH9O2RnpjFuYeumn6eM6S8SS+q0LiIiIY83PbBFnTdb3xBzANVJ05bziNwlsgbrMe7mdbPZ3LJVu9quaDT+fRb4ZuwVP79AJyeXeqiOsiorYkry6qPl+oOYA6iOn0z5SYsu9dT4/t3bxw3r0b194PgRPY8c/JOIvvvy/c8WvU1EQ/q07RThWbXrevHcyZcmD+3RIah7+8CJI3sf3PsHESUn3ekU4Rn9+7r5b03v0SHo+yUfEdHzo/oe3PtHRkZqpwjPXl2a1fnBdylf8EdWUt22Wd8QcwDVstTXHflnTx37cP6slq3bvfvB137+TWRyORENGzXJzy+wbYeuq37dterXXcEh4UR0cN+O12aMdnP3+PirH+d/+E1yYvztm9eIKCXpDhFtWLeiy1O9lv+09dkR44lo1pz3iGjcpBmrft31wy/bebw6Lt9BJPaRKSrNprpttl4JbV0AQKP2ftyFvh7+Le2c67zl82eOE9GceR/LZPKBQ0ZaJ/r6BalyswcMGdWmbSfrFFVu1meL3h707KgPPllORAX5KpPJFNo8gohSku4S0Zz/Le7ec0BVsyKxmIi69xpY1UKde94/TMoX1FPj9QGjOYDqVJpMSkG9jAZCQsOJ6IN5s/LzcqomJiXG6Q36sBatqqZs3bRGr6t86dX/Wf+07sZaF0hNvuPh5XNvxhFR/O3rRBTavGV91Gx1vig3p5JNF9Ah5gCq81nLLoFy+/poefCwsW/N+zjm4ulRg7vtit5knRh/6wYRhYZFVC12+sTh1pEdPb19rX/euR0rlcoCAoOJKDnpbouWbe9rNv72Df/ApgpFPd54e7k0vyFPPT8+xBxAdYr0lQaLuT5a5vF4oydM2/rnKV//oK8+mafVaqyDNRdXdzd3z6rFsjJTvXz8q/68dP5kSGi4QCAwmUwZqUlNgpvd12z8rRuhYfU4lCOitg5u/jJFvXZRtxBzANXZmpV4sVhVHy3r9ToicnXz6BrVy2g0ms1mIkq6G+fm/q/HWAqFIuvhNutFv9euXPDw9CaizPQUvUEf2CT0X20a9Gmpife1UOeecvF2FkvrtYu6hVMQANUJVTqq6uFGzssXTn/20dwRYyYTUfSW9T37DFYolESkVNjfvH78t/U/ikTi7j37e3j6tIrscPLYgYGDR5oMho8/nGOxWDQaDRGlJN8hoqb/Hs2JhCKZXHH00J9NQ8JKy0omPD+zzivXmU1bMu++0iSiBss2Fog5gOr0c/dXGw26ur5+QqfXKxR2Py773NHJ+dnhY6e9Mtc6/YWZbyYn31nx7cfOLm5RPfoR0TsLv/zkgzdnzxzn6ureNap3TmbG3YRb1gNzAoHAL6Dpvc3yeLzX3/rg+28//mLx/8JatKqPmEtQlxjZdiM8bt0HeIREdamDWILjO1Z3K0p8pMoGvsn3MWE0B/AIZ4tyJQJ+Hze/hy1w7vTxBXNf+u90iUSi0+keuMqaTXuCmtx/9qA+zJj8bGJC/H+nN28REXcr9oGr/LHvvKPTQ68TjLB3VdTPFTb1B6M5gEcoNxpWpdycGtD8YQtUVmqLCvL/O91g0ItE4geu4ubhJRI1xNPr8vNyDHrDf6fzeA/97Xt6+/L5Dx68XistUBsNQ72C6rrM+oWYA3g0s8VSbHjwuOyJsvD2+S9bdrNj2+NFEXMANbIs6fo4v9An+QidiSyOQon4IQO9xox9FQPYxNOuPj+mPPhg1pNAbTTkVlawMeMwmgOoBb3ZXGbU8errkSWNV5lRvzrl1qf19kCq+sbKbAawCTGf7yySRmcl2rqQBqW3mIv0OvZmHGIOoHb4PF6g0mFndrKtC2kgJwuztSZjO0c3WxfyWBBzALXTz81vgGeAUii6zLZnhdfWmcKcMqO+Sf08oKUhIeYAas1HqpDwBQIeb/7tc1qT0dbl1LGLxaqNGXfsReIuLp6T/cJsXU4dwCkIAOZ0FpPBZBby+K9e/ytQYT8jsKWQx7taWmAym9s7uZvJcqkoT8DjNfLPBov5REF2mUE30ju4wFC5Nzd1sGdgsMLB1lu3zmA0B8CchCdQCkVSgeDTFl2iXL0dRWJHkTi5ovRKSb69SCwXiC4Wq+rq87pTR3OSU+q2TetnMY9fZtAFyO2cxJIQhcPspq25lHEYzQGwxrRp01599dXIyEhbF8I+GM0BAMch5gCA4xBzAOzg4+MjELDptYGNB2IOgB2ysrJMJja9BLrxQMwBsINcLn/YY+CgethqAOyg0Wisb/+C2kLMAbCDk5MTRnPMYKsBsENxcTFGc8wg5gDYwdfXF2damUHMAbBDZmYmzrQyg5gDAI5DzAGwg1KpxCkIZrDVANhBrVbjFAQziDkAdrC3t8dojhlsNQB2KCsrw2iOGcQcAHAcYg6AHTw8PHDdHDOIOQB2UKlUuG6OGcQcAHAcYg6AHXCzF2OIOQB2wM1ejCHmAIDjEHMA7ODn54edVmYQcwDskJGRgZ1WZhBzAMBxiDkAdsALDBlDzAGwA15gyBhiDgA4DjEHwA54Tytj2GoA7ID3tDKGmANgB09PT5yCYAYxB8AOubm5OAXBDGIOADgOMQfADo6Ojjwez9ZVsBJiDoAdSkpKLBaLratgJcQcADvgeXOMIeYA2AHPm2MMMQfADhjNMYaYA2AHjOYYQ8wBsIOLiwtGc8zwcO4GoDHr16+fRCLh8/lFRUUKhUIkEvH5fJFItH37dluXxhpCWxcAANVRKBQZGRnWz1qt1vph5syZNi2KZbDTCtCoDRw48L6rgv38/EaPHm27itgHMQfQqI0aNcrHx+feKQMGDLC3t7ddReyDmANo1JycnAYMGFD1p5+f3/jx421aEfsg5gAau/Hjx/v5+Vk/DxgwwM7OztYVsQxiDqCxs7e379+/P4/HCwgIwFCOAZxphSeFhShdU55VWWFi4TN4/fv3dLkT27pr12u6ctKV27qcWrMXiZsqHZQCkU16x3Vz8EQ4UZAVnZ1UoteF2DmV6ittXc4TR8Djx6uLOzp5LAht3/C9I+aA+44XZO3MTh7jG8wnPK/Nlm6XF18qVi1t1V3csO/uQcwBx10oVm1Ij5/oF2rrQoCIKF1bfqogZ3nr7g3ZKU5BAMdty0p81quJrauAv/nL7NwkstOFOQ3ZKWIOuKzSbIovL7IX2ubINzyQlC+ILy9uyB4Rc8BlOZUVQQoHW1cB/+IqlhYbGvQsEGIOuIxHvDKD3tZVwL+YyKJt2AfnIeYAgOMQcwDAcYg5AOA4xBwAcBxiDgA4DjEHAByHmAMAjkPMAQDHIeYAgOMQcwDAcYg5AK5JvHV9w7ef3r58oT4aP3/0wKqP5qlLS4hIr6s8sOXX9Us+qY+O6hBiDuB+Rfm5S999fUbfjjMHdL54/GBJQf4bw3p+/96btW3HYNBfOHZAr6vpberq0pKYE4dqX+/9ju/adnDr+tKigsdv6r+2rvzm1P6dRoOBiMpLizd+91nsxdP10VEdwrsgAO733f9eS46LbdI8QiyVNg2PKMzLLVQxeT7agknDstOSVx26IJZIH7lwoSr77VEDvAKbtO/Rj1HV8FCIOYB/yU1PTY6LDYmI/OCnzdYpLh7ec79d7erpXdumtBUVNV/YqDcY8DCV+oGYA/hHzInD3817jYjuxl6d2CVs6twPfZsGL545kYj8Q8I+Xb+TiKb36aCtKB869eXTe3cWF+aNeHHWMxNe2Lz8qwvHD1RqKrz8mwx/4ZV23XvPGdm3uEBFRC/160REM9//4qmBQx/Wb2Fezluj+xNR+t34iV3CiGjpruMu7l4addmWlUsu/XVYW17u4es/YNyUp4eMtK5SzaxafuUj+zevTbsbxxeIgltEjH55TmCz8IykhDWfvZeZkmg0Gn2Dgoc8P71TrwE1aKyRwrE5gH84uLgFt2hNRHZOzm269nD19FY6OIW37/zfJXev/yk0sn3zyE5RzwzbtXbloe0bhSJxyw7dykuLDXodEUV26ymSSImofY++nfsMdPP2qaZfiUTWpmsPIpIr7Tv3Gdi5z0CJRGY0GD5//cWjf/wuEolDWrdTZWf+/OnCA1t+JaJqZtXKgS2/fjdvVsKNK55+QW6e3jfOny4vKSYiuZ2dKjsjoFlz36Dg1Du3vl/4ZvLtm7VtvPHAaA7gHyEt24x+ec6nsyY3bR7x9jerrBMnzZ7/7sRn71ty8pz3eo8Ya/2ckXyXiJ6b/nr3QcOMBoOFLEQ06c35F48dLNZVTl/wscLuEU8wVjo4Tpo9/9rZE65e3rMWf2udeGrfjuS42IBm4R+s2iSWyhJuXPnopfF//Lyi9/Cx54/se9gskVhSwy9bUpC/ZcU3PB7vf0vXtOzQlYiyUpN8ApsSkYu71w97z/B4PCLa//u6TUs/v3Bsf5PwlrXcnI0FYg6AiU59BlZ9btO1x+WTRzct/aysML/X8DFypX2ddBF78SwR9RjynFgqI6Jmrdp6BQTlpKWkJyZUM6tpeEQN279x8YzBoG/V+SlrxhGRNeOISF+pPbx90+mDuwuysyxkJqK8rIw6+VI2gZ1WACakckXV555DRz//1kKLhX7/4ZvZI/peP3uyTrooLykiIidXt6opdo7ORKQuK6lmVs3bLy3IJyJ3H///zlq64I3NK77WVqg79OofFtmRiHSVmsf+QjaDmAOoA/1GTly648iAsZM15aUrP3rHelmZlcVci1chm83mqs/W5CorKqqaUpKfR0T2Dk7VzKp5X3I7eyIqzs+7b7oqK+P62ZPObp5fbNo9Y8EnQyZNIyJWv9AZMQfwuPS6yqJ8lUxhN/GNd2Vypbq0pKKslIhkCgURZaenWC8Vrr4RqUJJRIW5OfpKrXX58LYdrUforOc0rp75Ky87w87R0S84tJpZVQ0aH9VjWGR7Irp29q+E2KvWKSl3bul1lZUaNRE5uLhIZHIiunvjKhGZTOZ717336xj0jf06GBybA3hc5w7vXb/k42at2up1Oq1G7RUQ5ODiSkQhrdpmpyV/PeclDz9/v6ahMxZUd1OUg7OLu49fXlbG3DGDZHZ2A0ZP6tp/yP4t6xNvXZ87dpCrp3fizWtENHLGm0KRqJpZRCSVyYno+rmTUYOGV9OjT2DT7oOfO7kn+uOZE3yahPB4vMykhClzP3hq4FA7J+eU+FufvPq8UCi6eeksEanSUy0WC4/Hs+6tXz97ss9z46wdFeRkZSbf9W0SUg+btm5gNAfwuOwcnT19A29fvpCZfLdd995vff33KdrRM99s07WHyWTISUt2cHZ+ZDuvfrQkoFl4aXFBcb5K6eAklkjnL18XNWh4paYi8eY1D7/AGe993nv4GCKqZhYRdeo9QG7nUJyfp60or77HF+d9NOblt9x8/LJTkwpVOWFtO/k2CRFLpG9+saJpeKvEWzdUmekvzvuoa/8hmgp1ZlICEUUNHCaTKzOS7xCRws6hw9P9lA6OSbdv1MWGrC88Vu9yA1QvVVP+QdyFl4PYeiUEJ8WWFeZUat4P69BgPWKnFaAhVGo0S+e/9rC5vYePbd+jb513mhB7dcea7x82d8rcDz18/Oq800YIMQfQEEwmQ+yFMw+b26pzVH10WlZUUE2nj9yl5QzEHEBDUNg5bDwX38Cdtu/Rt+E7bYRwCgIAOA4xBwAch5gDAI5DzAEAxyHmAIDjEHMAwHGIOQDgOMQcAHAcYg4AOA4xBwAch5gDLhPxeW7iR78KGhoSj9fQ/1EQc8BlPlJlfHmx3mKuwbLQQDI1ak+pvCF7RMwBx/V290vTlNm6CvhHuVHfzsmjIXtEzAHHvRncZld2colRZ+tCgIhoe1bSUy7e/jJlQ3aKpwcD91WaTVNiDj/l6m0nFHtI5WbswzY4vdmSpS2PLSsa6hU00COggXtHzMGTYkvW3eslBRaiTK3a1rUwUV5eLpPJhEJWPiPSWyZ3F8sHewWFKh0bvnfEHAA7TJs27dVXX42MjLR1IeyDY3MAwHGIOQDgOMQcADv4+voKBAJbV8FKiDkAdsjMzDSZTLaugpUQcwDs4OnpidEcM4g5AHbIzc3FaI4ZxBwAO/j4+GA0xwxiDoAdsrKyMJpjBjEHwA4eHh4YzTGDmANgB5VKhdEcM4g5AOA4xBwAO+CCEsYQcwDsgAtKGEPMAQDHIeYA2EEsFvN4PFtXwUqIOQB20Ov1eDokM4g5AHZQKBR8Pn6wTGCrAbBDRUWF2Yy3WDCBmAMAjkPMAbCDi4sLdlqZwVYDYIfCwkLstDKDmAMAjkPMAbADbvZiDDEHwA642YsxxBwAcBxiDoAd8AJDxhBzAOyAFxgyhpgDAI5DzAGwA860MoaYA2AHnGllDDEHwA54Qglj2GoA7IAnlDCGmAMAjkPMAbCDj48PTkEwg5gDYIesrCycgmAGMQfADrgLgjHEHAA74C4IxhBzAOyAY3OMIeYA2AHH5hhDzAGwA47NMcbDC24BGrM+ffrw+Xw+n19aWiqTyYRCIZ/Pd3Jy2rx5s61LYw2hrQsAgOqIxeK8vDzrZ4PBQEQ8Hm/AgAG2rotNsNMK0Kh16tTpvkNygYGBI0aMsF1F7IOYA2jUpkyZ4uXlVfUnj8fr2rWrv7+/TYtiGcQcQKMWEBDQqVOnqmPogYGBo0aNsnVRLIOYA2jspk6dah3QWSyWp556ytfX19YVsQxiDqCx8/f37969u8Vi8fHxwVCOAZxpBS6rMBlLDTpbV1EHej437Ni1y+27diVnh+zKCluX89h45C1RNFxvuG4OOGlbVuLO7GSjxczn8WxdC9zPR6aMLy/q4uz1RnBrpUBU390h5oCDlifdyNNrujh7OYsktq4FHkxvNufoNJvS49e27+MiktZrX4g54JrlSddLjfo+bn62LgRqZHH8pejOg6T8eryPDacggFPi1SUqnRYZxyLj/UNXJN+o1y4Qc8Apd9XFOBTHLq5i6aUiVb12gZgDTinQVXrLGu4UHjw+B6HYTSLXmI311wUuKAFOUZsMJgve8scyqZoyXn2eI8BoDgA4DjEHAByHmAMAjkPMAQDHIeYAgOMQcwDAcYg5AOA4xBwAcBxiDgA4DjEHAByHmIMnXeKt6xu+/fT25Qu2LoRMJtONC6f3b173wLkFuVnRPy8/tH1Tg9dVC+ePHlj10Tx1aYmtC/kXxBw86Y7v2nZw6/rSogLrnzvWrHipf6ek29U9GkiVkRZ35WLNu0i4fjknLeWRi5UXF305e9rh6AcHWfzVmB1rVmQk3al5vzWnLi2JOXHo8dvZuvKbU/t3Gg2GuiiqziDmAP4l8fb1irLSzOS7D1vg/JF9b43uH3PiSA0bXPvVhx/NnJCVmlh3NdaxQlX2a0O6/7HmB1sXUl/whBKAf5n+7id3b15tG9X7YQtoK9S1alBb0djfUGPUGwwGva2rqEeIOYB/fPra1Nsx54ho9ufL2/fom5+TteHbT+KuXOLx+U3CWkyasyAl7uaaz98nooNb1x/cut7dx2/J9sPVNLj604VnD+4mou/mvUZEPYaMnD7/Y72uctmC2Um3rmnUahd3r+6DRwx5foZA8PdTwtWlpYumj01JiHNwcu7c95nnps0SSx78qoTzRw/s/nVVdmqSVKmM7NZz7Ctv2Ts5E9GR6M37f19bmKdycffoPvi5oZNfqqbCwryct0b3J6L0u/ETu4QR0dJdx13cvTTqsi0rl1z667C2vNzD13/AuClPDxlpXaWaWY0TdloB/tGsVaSjq3vVnysXvXPl1DFPf/9mEW1S7tySKZRu3j5BzVsSkad/YOc+AyO79ay+wabhES6e3kTUrHW7zn0GNg2PICKxRFqQm+3pGxjconVRQd72n5Ye3Lq+ahWNuqyoIM+vaUhZSfHejT9/+86rD3xhy4Etv36/cHZ2ekqT8AiZTHFyT/TilydoKypuXjq77utFpUUFbbp0l8qVhars6iuUSGRtuvYgIrnSvnOfgZ37DJRIZEaD4fPXXzz6x+8ikTikdTtVdubPny48sOVXIqpmVqOF0RzAP0ZOfz0rOfHSX38fjM9ITCCiNz5d5urpU6nRSOVyZzePXkNHr4m72bpz90lvzn9kg72GjYm/FnM2N3vQuCnte/Stmv7Zhl08Ho+IUhNuL5w84tzhvYPGTbXOcnH3WrL9sEAgKMjN+nD6uNiLZ66ePt42qte9zZYWFmxZ8Y1Urlj8y3avgCCLxbJy0TtnD+7+a/c26wIdew6YsfBTIqrUaKqvUOngOGn2/GtnT7h6ec9a/K114ql9O5LjYgOahX+wapNYKku4ceWjl8b/8fOK3sPHnj+y72GzROJG+h41jOYAHiryqaeJ6Ks3Z5w9uFskqcvf8IVjBxfPnPDyoK4fvzyJiPKzM6tmCURC6w6sq6fP04NHENGtmPP3rX79wmmDQW/n6HR819bfln2xefmX1iOGSbdjIzo9JRAKTx/Yte7rRaqsDKlczqC82ItniajHkOfEUhkRNWvV1isgSKMuS09MqGbWY2+V+oKYA3ioafM+6j1ibE5G6g8fzn13wpC87Iw6aXbvxp+/Xzg7PTGhRbvOXfo+w+PxKrXaBy5p7+L6wJMepQX51nDct3mt9X9XTx+37g77BgW/s2S1u4/fkejNc8cM2Ll2JYMKy0uKiMjJ1a1qip2jMxGpy0qqmcWgo4aBnVaAhxJLZVPnfjho/Iu/fP7BrZizG7/7bM6Xf191YTHX7o0T9x5fO7RtExG9/+NGv+BQi8VyYk807yGvSy7MzSEiZzf3+6bLlXZE1LnPoFmLl/x3rRYdunzx295T+3as+3rx9p+Wtu4SFRTW8pEVmu/5RtbkKisqqppSkp9HRPYOTtXMemQXtoLRHAARkfFBV1QU5av0lVoPH7+xr84hopz0FCKSKeyqPpvNZqPxEa+kkikURJSdlkJE1us2tJoKInLx8iGi5LhYs8lkMv3TiFFvMJlMRJSbmX5y3w4iatWl+71ziSisbQciunzqWNLtWOv0lDu3dNq/D8PlZqYLBIKnh4yM6NiViFSZ6dVXKFUorZGqr9Raiwxv29F6hM6g1xHR1TN/5WVn2Dk6+gWHVjOrqsHGdnkKRnPwpJPK5ER0/dzJqEHD75u1deWS2Itnglu0zk5LJqLmbTsSUZPwlnyBIPbimXkTn9Wqy+cvX+fhF1BN+yEtI4/+8Xv06mUxJw7rdbovNu0Oi2x/5dSxRdPGePoH3Y45b43L3Mx0qVRGREX5uW+N6idTKHPSko0GQ+c+g5q1aktE1qNs8dcuVWo0PoFNowYOO7V/56LpY/xDmhuNhuyUxHGvvTNw7JTczPR3xgxo2rKNvaPTjfOnhGJJ0/BW1W8BB2cXdx+/vKyMuWMGyezsBoye1LX/kP1b1ifeuj537CBXT+/Em9eIaOSMN4UiUTWziEgqVxDR9bMn+zw3ri7+49QNjObgSdep9wC5nUNxfp62ovy+Wd4BTYUi8dUzf2krKvo+N2H8rP8Rkbu337R3F7t4eOWkJVvMFpH0EacmuvYf0m/UJLnSLjMxQWnvQERT5n7Qrnufovy8hBsxPZ597vk5CyQyWdzlv88z9Bs5USKR5qalOLt5jpg2a+YHX1qnh0V2tOZpbmYqEU1b8MmombPdvH3TE+MLc7LD2nYMCA4jIpPR0KJDl7SE2zcvnQ1sFv721yvdvH0fuRFe/WhJQLPw0uKC4nyV0sFJLJHOX74uatDwSk1F4s1rHn6BM977vPfwMdbDfw+bRURRA4fJ5MqM5Hq5I40x3gMvyQFgqeXJN0wWc2cnT1sXArXwyZ2YrR0HyAT1tXOJnVaAx5IQe3XHmu8fNnfK3A89fPwatqL7VWo0S+e/9rC5vYePvfeCPk5CzAE8lrKigtgLZx429787wg3PZDJUU2GrzlENW44NIOYAHkv7Hn03nou3dRXVUdg5NPIK6xtOQQAAxyHmAIDjEHMAwHGIOQDgOMQcAHAcYg4AOA4xBwAch5gDAI5DzAEAxyHmAIDjEHPAKfZCsZiHWxhZpqnSgXj12D5iDjjFXSLL1Nr+bnmouWKDLr9SK+PX4/85IeaAU8LsnHn1OS6AOpen03Zx8arXLhBzwClBcrsQpePu3FRbFwI1ojebf89MmNUkol57wdODgYO2ZSVeKs7r4OTuJVUIMbprlIoNuiKDbkNa/M4ug6X8+h1vIeaAm/4qyIrOSlTptFqjwdaf9tnVAAAgAElEQVS11A2jySQQ8Hn1eqy+oYTYOebptN1cvF4Oqt9xnBViDjhOa3rECwbZ4tVXX50xY0br1q1tXUgdsPBIXp/nHO6DU+/AcfX3IpUGxjeaJDw+Z75OQ8IpCADgOMQcADt4eHgIBAJbV8FKiDkAdlCpVCaTydZVsBJiDoAd/Pz8MJpjBjEHwA4ZGRkYzTGDmANgB19fX4zmmEHMAbBDZmYmRnPMIOYA2EEmk/Fw4xojiDkAdtBqtbhniRnEHABwHGIOgB1wQQljiDkAdsAFJYwh5gCA4xBzAOzg5ubGr+fHT3IVthoAO+Tn55vNZltXwUqIOQDgOMQcADvI5XJcHswMYg6AHTQaDS4PZgYxB8AOPB4PozlmEHMA7GCxWDCaYwYxBwAch5gDYAelUonr5pjBVgNgB7VajevmmEHMAQDHIeYA2AEvMGQMMQfADniBIWOIOQDgOMQcADvgsZqMIeYA2AGP1WQMMQcAHIeYA2AHPKGEMcQcADvgCSWMIeYA2MHX1xenIJhBzAGwQ2ZmJk5BMIOYA2AHV1dX3LrPDLYaADsUFBTg1n1mEHMA7ODs7IzRHDPYagDsUFRUhNEcM4g5AHbAzV6MIeYA2AE3ezHGwwWHAI3ZoEGD8vLyqn6nPB7PbDb37dv3iy++sHVprIHRHECj1qpVq6q3F1pv9vL29p48ebKt62ITxBxAozZhwgQvL697p7Rp0yY8PNx2FbEPYg6gUYuIiGjVqlXVTquHh8e4ceNsXRTLIOYAGruxY8d6enpa30gdGRnZokULW1fEMog5gMYuIiIiMjKSiDw9PcePH2/rctgHMQfAAqNHj3Z2dm7dujWOyjGAC0qATbZk3j1XlMMjXqK6xNa1NDS9wSAUCvlP2JM1ZSKRnC9sYe88yT/MUyJn1ghiDljjlWt/NVM6ekjl3lLFk/Vbf5LxeKUGXb6u8qAq7cPmHZvbOTNpAzEHrPDytb/aOrpF2LvYuhCwmTVpt2cEtuzg5F7bFXFsDlhga+bdMDsnZNwTbop/8w0Z8QzGZYg5YIHzxSo3sczWVYCNCXg8rcl4p7y4tisi5oAdvGQMDz8DlzRVOGRoy2u7FmIOWCC5ooxwDBmItCZThclY27UQcwDAcYg5AOA4xBwAcBxiDgA4DjEHAByHmAMAjkPMAQDHIeYAgOMQcwDAcYg5AOA4xBwAcBxiDqBGNBXq80cPnD2055FLnj96YNVH89Sl7Hu+ccKNKzvXrjTodbYupI4h5gBq5HbMue8Xzr529sQjl9y68ptT+3caDYZ6qkSVkRZ35WJ9tPzVnJe2/7TUaKj1vfGNHGIOgE3OH9n31uj+MSeO2LoQNkHMATQWibeuP3IZbYW6QWrhFKGtCwCoe/t/X/fbsi/GvvLWyb078nKyXD29eg0bk5+VeeXMcXVpSUhEmylzP/Tw8bMufO3sieifl2cmJohlsoiO3ca9NtfF3cs6Kys1afP3X8VduSAQiNy8fe/twmg07l7/04k90SUFec5unlHPDB/y/AyhsHY/qIQbV7b++G3qndsisSi4ZeTV08cnvTm//+jnrQf4dv+6Kjs1SapURnbrOfaVt+ydnE/v37Xm8/eJ6ODW9Qe3rnf38Vuy/XD1XRyJ3rz/97WFeSoXd4/ug58bOvklIjKZTPt/X3d6747crAw7e8dWXaLGvDzH3unvt8n8/sNXMSeOGHT64JatJs1e4BUQZJ1eWlS4ZeWSq6ePVlZofJqEDJ40vXPvAUSUmnB74eQRA8ZOTr1zOyX+ptLesV1Ub1dvnxN7/sjPzvDyCxr32tyWHboSkV5XuWzB7KRb1zRqtYu7V/fBI4Y8P0MgENRqozGA0Rxwk8Vi2bziazdv35YduuSkpWxa+vmxXVvC2rTzDQqOvXDm+/dmWxeLOXHom7dnpiXEhbSKtHdyPn9k3+KZEzXqMiLKzUxfNGP8tTN/SeUKL//AzOSEextfvmB29Oplukpt0xatNRXl0auXrVo8r1YVZiQlfPb6C/FXL/kENlXYO149fVwgFEY+1ZOIDmz59fuFs7PTU5qER8hkipN7ohe/PEFbUeHm7RPUvCURefoHdu4zMLJbz+q7uHnp7LqvF5UWFbTp0l0qVxaqsq3FL5v/xu/ff5WXkxkUGi4Siy8ePUD3vCrt2I4tLu5eQrHoxvnTX86ZrtdVEpG6tGTRjLEn90TLlfZB4RFZyXe/Xzj72K4tVWsd+P1XrVrd8en+ZSXFh7Zv/G3ZFy7uHq06RaXdjftu3qzi/DwiEkukBbnZnr6BwS1aFxXkbf9p6cGt62u10ZjBaA44q9uAIS9/8BURffnmtBvnT4+c/vrgSdONRuPs4b1S4m4V5auc3Tw2LfvSYrG8+uFXnfsMMplM37z90o3zp4/+sWXI89O3rlyiKS/tNmDI9PmfCkWiU/t2rVr8P2vLl08evXzySECz8Pd/3CiRyTUV6vdfGHnu0J5nJrwQ2Kymr4s+f3ivQVc5cNyUCa/PM5lM778wKi3hdn5WpkQi27LiG6lcsfiX7V4BQRaLZeWid84e3P3X7m0Dx07pNXT0mribrTt3n/Tm/Ed2kZGUQEQdew6YsfBTIqrUaIjo8skjl08ecXbzfP+nTa6ePtZBq73jPy8GnL9iffPIDpUazfsvjMxOS467cql1l6gda3/Iy8roNXzM1Lkf8ni8jKSEhVNGbF35bY/BI61refgFfLB6s1gi9Q5sumXlN627RM1dspqIli+cfeHogbgrF7r2H0JEn23YxePxqsaA5w7vHTRuKqP/vLWAmAPOcvXwtn5w8fQmIkdXdyISCoUevv4lBXmlBfmGysr87Ex7R6dOvQcSkUAgiBo0/Mb50/HXLw22TLt+7gQRjXpptlAkIiKZ4p+XUVw5dYyIpHJ59Orl1ikSiYyIkm/H1jzmzBYLEfF4fGvXfAGfiIwGw/ULpw0GvaOb+/FdW61LWo/HJd2Ore0WiOj0lEAoPH1gl1gqGTjuBet++pVTx4mo78gJ1owjIp/ApveuZf0KUrm8VZeo7LTkvKyMqq9cqdFsXv6ldTGZQqkuLcnLTLf+ae/kIpZIicjVy7q1PazTvQKaEFFxYb71zwvHDh7etiE7PcWg0xFRfnZmbb8UA4g5eOJYRxMWC5WVFhORvYsb7/9fZW/n6EREFaWllRq1TqvlCwRVWXCvksI8IrpzLebOtZh7p4vE0pqX0eOZEYe3b9z32y+pCXHairKUuFv2zi7BEa2P7dhq/f3v27z23uWtIVIrvkHB7yxZvfbrRUeiNx/buXXEi7OGTX3ZWry7r98jVxeKxERkNOqJqLggn4jOHtx93zJiqUSn01bTyP9vbQsR7d348+YVX8sUdq27RMkUyr/+3FaprW7duoKYgyeXvYMTEZUVF1ZNKc7PJyKlo5NMYSeWSvWVlaVFhQ7O978fVq60I6Kp7yzqPXwM4949/QP7j5m8a+3Ku7FX+XxeWGSHCW/MU9g5WBvv3GfQrMVLHrauxWyuYS8tOnT54re9p/btWPf14u0/LW3dJUqutCeikoK8WlUrVyrLinRfbt7nHdjkvlnlNb4Q+tC2TUT0/o8b/YJDLRbLiT3RPEtDvMoIpyDgyeXu6+/i7lVWVHj55FEiMhj01mPqLdp1JqKAkOZEtP2npUajkYj0lZVVK4a16UhEB7f8WlZcZJ2ScP3yfY0bDPrqe8/NTP9z3Y9BYS1+OnxxzbGrC3/YEBTagojC2nYgosunjlXtpabcuaXTaqyfZQo7IspJTyEis9lsra36XgQCwdNDRkZ07EpEqsz05m07EdGh7ZuspwWIKCH26iO3VfPIDkS0Y+0P1u9lNBgY7ERrNRVE5OLlQ0TJcbFmk8lU+9d0MYDRHDy5eDzeqJmzf/zof8sXzg5u2aYgN7sgJ8vD1//pZ0cR0Yhps75448Xju7ZePnXUxcMzI/GfM61Rg4Ye3r4xKzVpzsg+vkEhZcVFedkZi9dFW3NKKlcQ0fWzJ/s8N66a3ksKVBaLRV1WumfDz6rMNB7xgiMiewwe4RPYNGrgsFP7dy6aPsY/pLnRaMhOSRz32jsDx04hoibhLfkCQezFM/MmPqtVl89fvs7DL+BhXeRmpr8zZkDTlm3sHZ1unD8lFEuahrdycHY5tH1jVvLdt8f09wkMVpeW5GVnPHCYdq/hL7x67eyJc4f23L583t3bT5WRyhMIvo0+Uqtd6bDI9ldOHVs0bYynf9DtmPPWpC4rKbr3BEh9wGgOnmhPDRw6a/G3PoHBiTevadTqrv2HLPhhg0yhIKKIjt1mLV7i0yREU16mKS9v3aV71VoSmXzByg09h44WS2XJcbGVlZrOfQYp7Oytc6MGDpPJlRnJd6rvOrhlZIen+xUX5EevXnZ6/65T+3eu/fKD6J+XEdG0BZ+Mmjnbzds3PTG+MCc7rG3HgOAw61ru3n7T3l3s4uGVk5ZsMVtEUkk1XZiMhhYduqQl3L556Wxgs/C3v17p5u0rlsoWrljfc9hoqVyRdjdOr6/sNmCIRC6rvlrfJiHv/bipTdceem1lclysVK7s1v/Zmu87W02Z+0G77n2K8vMSbsT0ePa55+cskMhkqv8/iVF/eJYG2TcGeBzDzu97o2krmYBrOx93rsc4u3u5efkQUcyJw9/Ney20Tfv3Vm60dV2N135VeqSj6zCv6gae/8W1fzcAjcfRHVtiThx64CypTDF40rTP35hm0FU6u3kKxaKCnCwiatU5qg67eOOzZYwK5xrEHEB9yU5Nir1w5oGzZAq76Qs+HjzxxUvHDqoy00USadMWrXsNGx01aHgddsGoag7CTiuwAFd3WqG2mO204hQEAHAcYg4AOA4xBwAch5gDAI5DzAEAxyHmAIDjEHMAwHGIOQDgOMQcAHAcYg5YwEem4PPwbxVIJhSIebV+Exj+6QALmCyWAn1DPE0bGrlMjdpDKq/Bgv+CmAMWaOPoWmyorMGCwHFCHj9IXutHEiDmgAVmBrbcmplo6yrAxvap0to7uTvX5r1CVnhCCbBDqVH/wuWjk/xDPSW13mcBttOZTQdUaWF2zpP9wxisjpgD1igx6JcnXT9flBvp6FpQ7UvzOKlSpxOJRAL+k7UHJhII83VaO6FoiGfQs15BzBpBzAHL6MymFE2ZsZZvIeCAL774YsSIESEhIbYupEFZiOcmkbqJZYL/f5cuA3hOIbCMhC8IUzrZugobkKiKAvmSlvb3vzQWHunJGgADwBMIMQfADjKZjPcYO25PMsQcADtotVocSWcGMQfADt7e3gJBre9zAsQcAGtkZ2ebTCZbV8FKiDkAdvDw8MBojhnEHAA7qFQqjOaYQcwBAMch5gDYQS6X85+wO73qCrYaADtoNBrzk3eLW51AzAGwAy4oYQwxB8AOuKCEMcQcAHAcYg6AHfz8/LDTygxiDoAdMjIysNPKDGIOADgOMQfADp6enthpZQYxB8AOubm52GllBjEHAByHmANgBycnJ9zsxQy2GgA7FBcX42YvZhBzAMBxiDkAdpBIJHjlDTOIOQB20Ol0eOUNM4g5AHbACwwZQ8wBsANeYMgYYg4AOA4xB8AOeKwmY4g5AHbAYzUZQ8wBAMch5gDYAU8oYQwxB8AOeEIJY4g5AHbw8PDAaI4ZxBwAO6hUKozmmEHMAbCDQCDAXRDMIOYA2MFkMuEuCGYQcwDAcYg5AOA4xBwAO7i5ueEh6cxgqwGwQ35+Ph6SzgwPBzUBGrN27doRUdU5VusPtk2bNmvWrLF1aayB0RxAo9ahQ4d7/+TxeI6OjtOnT7ddReyDmANo1MaPH+/g4HDvlNDQ0M6dO9uuIvZBzAE0at27dw8NDa36097efuLEiTatiH0QcwCN3YQJE+zt7a2fQ0JCunbtauuKWAYxB9DYPfXUU8HBwRaLxd7efurUqbYuh30QcwAs8Pzzz9vZ2TVr1gxH5RjABSXABQaLeUvm3aSK0kK91ta11JeU5GQ3N3elndLWhdQLb6lSJhC2cXDr4epd540j5oD17qpL3rhxsrurj4dELhcIbV0OMMHn8bIrKypMBoPZ/EFYx7ptHDEH7HazrHBl8s1J/qE1WBZY4FRBtoks85q1q8M2cWwOWMxksXyXdH2Mb4itC4E6E+XqzSPenzkpddgmYg5Y7HJJnkIgFOOGdm4JUtgdzsuowwbx7wNYLEOrDpDb2boKqGPeUoXBXJePg0fMAYuVGnRGPLSDc4Q8frKmrA4bRMwBAMch5gCA4xBzAMBxiDkA4DjEHABwHGIOADgOMQcAHIeYAwCOQ8wBAMch5gCA4xBzAMBxiDkAW0q4cWXn2pUGvc7WhdSX80cPrPponrq0xIY1IOYAHk2VkRZ35WJ9tPzVnJe2/7TUaDDWR+OPo66+8taV35zav9NoMNRFUQwh5gAe4fyRfW+N7h9z4oitC2k4HPvKiDl4oiXeuv7IZbQV6gappRHh2FfG+0HgyZJw48rWH79NvXNbJBYFt4y8evr4pDfn9x/9vPUo0u5fV2WnJkmVyshuPce+8pa9k/Pp/bvWfP4+ER3cuv7g1vXuPn5Lth+uvosj0Zv3/762ME/l4u7RffBzQye/REQmk2n/7+tO792Rm5VhZ+/YqkvUmJfn2Ds5W1f5/YevYk4cMej0wS1bTZq9wCsgyDq9tKhwy8olV08frazQ+DQJGTxpeufeA4goNeH2wskjBoydnHrndkr8TaW9Y7uo3q7ePif2/JGfneHlFzTutbktO3QlIr2uctmC2Um3rmnUahd3r+6DRwx5foZAIKim/od9ZY26bMvKJZf+OqwtL/fw9R8wbsrTQ0ZaV6lmVmOA0Rw8QTKSEj57/YX4q5d8Apsq7B2vnj4uEAojn+pJRAe2/Pr9wtnZ6SlNwiNkMsXJPdGLX56grahw8/YJat6SiDz9Azv3GRjZrWf1Xdy8dHbd14tKiwradOkulSsLVdlEZLFYls1/4/fvv8rLyQwKDReJxRePHiDeP2sd27HFxd1LKBbdOH/6yznT9bpKIlKXliyaMfbknmi50j4oPCIr+e73C2cf27Wlaq0Dv/+qVas7Pt2/rKT40PaNvy37wsXdo1WnqLS7cd/Nm1Wcn0dEYom0IDfb0zcwuEXrooK87T8tPbh1ffVf4YFf2WgwfP76i0f/+F0kEoe0bqfKzvz504UHtvxa/axGAqM5eIKcP7zXoKscOG7KhNfnmUym918YlZZwOz8rUyKRbVnxjVSuWPzLdq+AIIvFsnLRO2cP7v5r97aBY6f0Gjp6TdzN1p27T3pz/iO7yEhKIKKOPQfMWPgpEVVqNER0+eSRyyePOLt5vv/TJldPHyLKSk2yd3SuWmv+ivXNIztUajTvvzAyOy057sql1l2idqz9IS8ro9fwMVPnfsjj8TKSEhZOGbF15bc9Bv89UPLwC/hg9WaxROod2HTLym9ad4mau2Q1ES1fOPvC0QNxVy507T+EiD7bsIvH41WNAc8d3jto3NRqvkJo6/b//crnDu9JjosNaBb+wapNYqks4caVj14a/8fPK3oPH3v+yL6HzRKJJY/9H60OIObgCWK2WIiIx+MTkUAg4Av41sHI9QunDQa9o5v78V1brUtaD04l3Y6tbRcRnZ4SCIWnD+wSSyUDx73g4eNHRFdOHSeiviMnWDOOiHwCm967VmCzcCKSyuWtukRlpyXnZWUQ0ZVTx6xBuXn5l9bFZAqlurQkLzPd+qe9k4tYIiUiVy9vInJ09bBO9wpoQkTFhfnWPy8cO3h424bs9BSDTkdE+dmZDDZd7MWzRNRjyHNiqYyImrVq6xUQlJOWkp6YUM2spuERDPqqc4g5eIL0eGbE4e0b9/32S2pCnLaiLCXulr2zS3BE62M7tlp///s2r713eWuI1IpvUPA7S1av/XrRkejNx3ZuHfHirGFTXy4pzCMid1+/R64uFImJyGjUE1FxQT4RnT24+75lxFKJTqetphHr2M36Cua9G3/evOJrmcKudZcomUL515/bKrXVrfsw5SVFROTk6lY1xc7ROSctRV1WUs0sBh3VB8QcPEE8/QP7j5m8a+3Ku7FX+XxeWGSHCW/MU9g5yJV2RNS5z6BZi5c8bF1Ljd+t06JDly9+23tq3451Xy/e/tPS1l2i5Ep7IiopyKtVtXKlsqxI9+Xmfd6BTe6bVV7jq20PbdtERO//uNEvONRisZzYE82r8Rvo7/3Kdo7ORFRWVFQ1pSQ/j4jsHZyqmVXDjuobTkHAEyQ3M/3PdT8GhbX46fDFNceuLvxhQ1BoCyIKa9uBiC6fOla1l5py55ZOq7F+linsiCgnPYWIzGaz0fiIS3lzM9MFAsHTQ0ZGdOxKRKrM9OZtOxHRoe2brKcFiCgh9uojq20e2YGIdqz9wWDQW3euGexEazUVROTi5UNEyXGxZpPJZHr0pcj//crhbTsS0al9O6w3bFw981dedoado6NfcGg1s6oatH4FW8FoDp4gJQUqi8WiLivds+FnVWYaj3jBEZE9Bo/wCWwaNXDYqf07F00f4x/S3Gg0ZKckjnvtnYFjpxBRk/CWfIEg9uKZeROf1arL5y9f5+EX8LAucjPT3xkzoGnLNvaOTjfOnxKKJU3DWzk4uxzavjEr+e7bY/r7BAarS0vysjMeOEy71/AXXr129sS5Q3tuXz7v7u2nykjlCQTfRh+p1a50WGT7K6eOLZo2xtM/6HbMeWts5Wame/r6V7PWf79y1/5D9m9Zn3jr+tyxg1w9vRNvXiOikTPeFIpE1cwiIqlcQUTXz57s89y4mpddtzCagydIcMvIDk/3Ky7Ij1697PT+Xaf271z75QfRPy8jomkLPhk1c7abt296YnxhTnZY244BwWHWtdy9/aa9u9jFwysnLdlitoik1Z09NBkNLTp0SUu4ffPS2cBm4W9/vdLN21cslS1csb7nsNFSuSLtbpxeX9ltwBCJXFZ9tb5NQt77cVObrj302srkuFipXNmt/7M133e2mjL3g3bd+xTl5yXciOnx7HPPz1kgkcniLp+vfq3/fmWxRDp/+bqoQcMrNRWJN695+AXOeO/z3sPHWI9gPmwWEUUNHCaTKzOS79Sq7LrFs9R4Rx2gsfkl7XaBTtvd1afmq9y5HuPs7uXm5UNEMScOfzfvtdA27d9bubE+y4Raez/uwqFuQ+uqNey0whMk6faNz9+YZtBVOrt5CsWigpwsImrVOapWjRzdsSXmxKEHzpLKFG98tqyOiq1HlRrN0vmvPWxu7+Fj2/fo27AV1S/EHDxBvPwDB0988dKxg6rMdJFE2rRF617DRkcNGl6rRrJTk2IvnHngLOuR+8bPZDI87CswyP3GDzutwGIMdlqBFep2pxWnIACA4xBzAMBxiDkA4DjEHABwHGIOADgOMQcAHIeYAwCOQ8wBAMch5gCA4xBzwGJ84gl4+DfMQQqhqA5vz8I/EWAxZ7G01KbPa4T6UGbUi3l8Xg2WrCHEHLBYE6W92oSY45oCfWULe5c6bBAxByzW0s5FxBMkqBvLq1WgThxWpU/wa1aHDSLmgN0+a9nlYrHqVnlRDZYFFliTdvu1pq2bKhzqsE08iAm44IO4C6macneJTMoX2LqW+mI0mQR8Aa8OD1k1JnKhKKWiVCoQjvEN6ebsVbeNI+aAI7IqK5IrSgv1lbYupL789ttv3bt39/X1tXUh9UImEPrKlM3tnPhU90GOpwcDR/hIFT5Sha2rqEd74lK79Xom0qu6l4HBA+HYHABwHGIOADgOMQfADnK5nM/HD5YJbDUAdhCLxbYuga0QcwDsUFJSYjabbV0FKyHmANhBJpPZugS2QswBsINWq7V1CWyFmAMAjkPMAbCDp6enQMDZW9nqFWIOgB1yc3NNJpOtq2AlxBwAO9jb2+O6OWaw1QDYoaysDBeUMIOYAwCOQ8wBsIOvry9OQTCDmANgh8zMTJyCYAYxBwAch5gDYAfstDKGmANgB+y0MoaYAwCOQ8wBsIOPjw92WplBzAGwQ1ZWFnZamUHMAQDHIeYA2MHd3R33tDKDrQbADnl5ebinlRnEHABwHGIOgB3wAkPGsNUA2EGj0WCnlRnEHAA7ODo6YjTHDLYaADvgPa2MIeYAgOMQcwDsgCeUMIaYA2AHPKGEMcQcADvIZDIej2frKlgJMQfADlqt1mKx2LoKVkLMAQDHIeYA2MHR0RE7rcwg5gDYoaSkBDutzCDmANjBxcUFd0Ewg60GwA5lZWUYzTGDmANgB4PBgJhjhocNB9CYtW3bloiqTj5Yf7BNmjTZvn27rUtjDYzmABq1Zs2a8e7B5/Pt7OymT59u67rYBDEH0KiNGzdOIpHcO8XHx6d///62q4h9EHMAjdrQoUN9fX2r/pRIJOPHj7dpReyDmANo7CZOnFg1oPPz8xs8eLCtK2IZxBxAY/fss89aB3RisXjixIm2Lod9EHMALDBhwgSxWBwQEIChHAO4oATgXy4Uq5IrSosNusb2y9izZ0+LFi2CgoJsXci/OIjEbhJZhL2rt1Ru61oeCjEH8DeNyTjnxil7kcRRJFYIRfhh1ISEz8/QqPUWU1dnrxHeTW1dzoMJbV0AQKOgNRnfuXmmn4e/t1Rh61pYJtLBjYi2ZyfaCUV93f1tXc4D4NgcABHR/26d7e7ijYxjbKR3cHR2UmxZoa0LeQDEHADdVZdUGA3+cjtbF8JuHZw8tmcl2rqKB0DMAVCypswH47jH5iNVZmnVtq7iARBzAFSs14nxbsDHJhcIVTqtrat4AMQcAHAcYg4AOA4xBwAch5gDAI5DzAEAxyHmAIDjEHMAwHGIOQDgOMQcAHAcYg4AOA4xBwAch5gDAI5DzAEAxyHmAIDjEHMAHGc0GFLu3LJ1FbaEd0EAMHT97Mmda39IvRuvtHPoNvDZg1s3PDf9tcETXtz/+7pNSz8fOvmlUTPfJCJtRcX0Pu3snV1+2HuGiBcyq2sAACAASURBVIxG4+71P53YE11SkOfs5hn1zPAhz88QCoVENL1PB21F+dCpL5/eu7O4MG/wpOkHt/xKRN/vPi1TKIhIU6F+ZVA3oVC4Ys8piay6N2md3PvH3t/WqjLTnVzdnN09U+Jvzlu2tllEZPW9h0S0dvP2vXzymL6yslmryOffWuju7deAW7S+YDQHwMTZg7u/emvG3ZvXnN3cHVxc92xYbdBVPnIti8WyfMHs6NXLdJXapi1aayrKo1cvW7V43r3L7F7/U2hk++aRnXoPH9Op10CdVnvh6D7rrJgTR4x6XYen+1WfcX/9ue2nj+fnZaQ1i2ij01TcuRbj7uMfGBL2yN5vnD997vD+Vp2jfJoEXzt74pu3ZhqNxsfYSI0FRnMAtWYw6Dcs/YyIxs6aO3jCi0T02/Iv9/32yyNXvHzy6OWTRwKahb//40aJTK6pUL//wshzh/Y8M+GFwGbh1mUmz3mv94ix1s9PDx11cu8fJ3ZHP/3sKCI6f3gPEUUNHFp9Lyf27iCi1z75rm1Ur5KC/NkjeuWkJfP4/Jr0vnjNVg+/ACJ6b+pzKfG3km5dC23d/rE3mI0h5gBqLSX+VnlxkYun9zPjX7BOkchkNVnxyqljRCSVy6NXL/97RYmMiJJvx1YFTac+A6uWbxYR6RsUfPfmtazUJDtHp5uXzrm4ezVv1+kR3ZjNRMTj84lIJBFbLBaT0WgymmrSu4uXj/VDYFiLlPhbqqxMxBzAk6isqJCIPHwDeDxerVYsKcwjojvXYu5ci7l3ukgsrfoslf/r5Ts9hozctOzzE7uj3b39zCZT1wFDHtlp31ET7968tuL9Oc3bdspISjAZjRGduknl8pr0XkUslhKRyaCv1RdsnBBzALUmVyiJqKww/4Fz+Xw+EZktlgesqLQjoqnvLOo9fEwN+3pq4NCtK5ec2r/T08ff+ucjV+nS95ljO7fEX710K+acSCzp2n/IxDfeZdY7N+AUBECtBYaFi0TizJTEC8cOWKcY7xn12Du5EFFq/N/XcJw7sqdqVlibjkR0cMuvZcVF1ikJ1y9X35edo1O7Hr3Li4vu3rwW1LyFT2DTR5Z3at+O+KuX+o2c+Mvxa6sOXnjlw6/snZyZ9c4NGM0B1JpcaT9g3OTd61cvXzA7OqCJSCxJuxtXNTe0TTuhWBJ78cz/xj1DRFmpSVWzogYNPbx9Y1Zq0pyRfXyDQsqKi/KyMxaviw4KbVFNd537PHP+yH4iiho4rCblFapyiSgzNWnryiV5OVlyuaJt995tuvZg1jsHYDQHwMTomXPGvvKWu7efKiu9tLjQO6BJ1SxnN89ZH33jHdBElZ0pEImen7OgapZEJl+wckPPoaPFUllyXGxlpaZzn0EKO/vq+wpv35mIBEJh5z7P1KS2qEFDA0NbxF+58Of6n84f3nts19av33rp+vlTzHrnAJ7lQUcQAJ4ov2feTawo6ePG/FLY6J+X71izour6krqVEHv1oxnjIp/q+dZXK2u4yuWTR5u36yRXKM1m82/Lvzjw+6/DXnhl5PTX67y2e+nN5q/uXtndZXC99sIAdloBGq/0u/HfvP1yYV6OUCQaNvXlqulHd2yJOXHogatIZYrmbTusX/KJQCh0cfcy6PXFBSoejxfRsWsDFt64IOYAGi+9XqfTadtG9Ro25ZUm4S2rpmenJsVeOPPAVWQKuzEvz+k5dPSN86cK83IU9g7tuvceOG4KBy5/Yww7rQB1sNMKjXmnFacgAIDjEHPwpIuLi7t+/brZjN0azkLMwZPo2LFjn332WVFREREtX768tLS0lndtAZsg5oD7dDodEW3ZsmXatGnx8fFEFB8fHxISYm9vT0Q//PBD9+7da3t3KrAIYg44qLi4OC8vj4h++eWXPn363Lx5k4icnZ1fffXV0NBQInrllVdGjhxpfZwkcB5iDrjAZDJdvnz51q1bRLRixYpRo0alpqYSUdeuXbdt29auXTsi6tu3b2RkJEZtTyDEHLBVcXHx9u3bjx49SkRr165dtWqVded04sSJR44c6dixIxGFhYU5OTnZulKwMQzagU1yc3N//fVXFxeXadOmxcTEJCYmDh06lIimTZs2bdo06zIODg62LhMaF8QcNF4qlcrJyYnP57/++usmk2nVqlVFRUVBQUHdunWz7oT27dvX1jUCCyDmoBHRaDQ3btwIDw+3t7efOnWqSqXasmWLXC6fPHlyq1atiCg8PDw8PNzWZQLL4Ngc2Fh8fPy2bdusn8eOHbthwwbrDYjffvvtvn377OzsBAJBp06dZDV72QIzjiKxEXc9Pjat2eglre6VY7aC0RzYQExMzNGjR4cPH96sWbPVq1f7+f19M+mff/5ZtYyjo2OD1ROkcNieldhg3XFVllbtKVXUYMGGhpiDeqfT6SQSydGjR7dt2zZixIh+/folJiYGBQX5+voS0TfffGPrAilU6SgTiLIqK3wa5a+ULa6VFLwQ2BgPKeAJJVD3jEajWq12dHTctGnT5s2b33zzzd69e58+fVoikURGRjbOi3IrTIa3Y8/09/D3lDTG3a7Gb1tWYm8330GegbYu5AEQc1A3UlJS9Hp9aGjohg0bVqxYsWzZso4dO964ccPNzc3Ly8vW1dVIudEw+8ZJd4ncUSS2E4mr/2WUlJQ05G5145SZlSni8QvJZOBRCF/6tNCOx+O1b9/oHmyHmAOGDAbDhQsX9Hp9r169/vjjj99+++2VV17p1atXXl6eu7u7ratj7mxRTqK6tMige+AbCK2uXbsmEolatGjQN8UcOnSoefPmVccxa0Kn0507d+7pp5+uj3oOHz5cUVHBU2uovMKSmikoKhMKhTwez2g0Hjt2rD56ZAwxB7WgVqsPHDhgMpnGjBlz/PjxXbt2Pffcc1FRUUajsXHuitY5lUrl4eGxZ8+ewYMb9OGRu3fv/uyzzwIDA3/77bdarXj69On09PTx48fXeUl379595513MjIy7pvu4eGxd+/eOu/ucSDm4BFKSkrWr18vFApfeeWVmJiYI0eO9OvXr23btrauq6GZzeZ333139OjR1jtkG9jYsWMTExPFYvH8+fMbOGGrsXXr1v9r774DmrraBoCfQMhmr7BkOEGl4ALBLdY9UARcrVtbsdWqfVV8tWptta+rjipWUZGiIKgodQEKuFBBEVAQZC+BsEMSMr8/rh9aS0ICN/dmnN9f8Y5zHoYP5557xokTJ1paWtqPEInE1NRUXIPqABw3B3WgtbV1y5Yt3333HQCgrq7O0NBw0qRJAIAhQ4Zs3rxZC3OcUChMTk6eMGECLjnuxo0bpaWlAAA+nx8ZGdmFEnbt2qWMBo2/v//w4cM/PcJkMisrK1GvqJtgaw4C9fX1JiYmQqFw9erVZWVld+7caWpqevr06aBBg8zMzPCODmc8Hm/Dhg2HDh0ikUh4xTB//vy8vDzkM41GCw4OnjhxokIlvHr1KiEhYcOGDajHJhQKAwICSkpKAAAkEunx48d+fn4+Pj6rV69Gva4ug2lOS717987Ozo5MJvv7+zc0NMTHx4tEoszMTFdXV11dXbyjUyGXLl1ycHDw9PTEK4Dr16/v27cPWXwF4eLiEhYWhlc8/5aWlrZt2zYWi5WWloYcSUlJGTVqVGpqav/+/fX19fEOED60ag2hUJiWltbY2AgAmDNnzrZt28RiMQDg4MGD8fHxAABdXV13d3eY4xCNjY3BwcFIpxiOOQ4AEBERwePxPj1SWlp6927Hm7TKduHCBTabjV5oHwwZMmTGjBkUCqX9yKhRo5B5LNOnT1dGjYqCrTlN1tzc/OzZM2dnZxsbm6+++opGo+3du9fIyIjH4336Swn9m6+v7549e1RhmQAPDw+BQEAgEAgEAvKXiUAgODg4xMTEKFpUdnb2/v37z507p5xIO8bhcOrr6+/cubNs2TIs6/0UTHOahsVipaSk9O3bt3///sHBwUKhcOPGjebm5njHpR5aW1tfvnw5YsQIvAPpwPLly9esWePu7t6dQoqKikgkko2NDXpxyeWPP/4oKCjAa2KfVox10ngsFuv27dt2dnajR4+Ojo6uq6vz8vICAOzZswfv0NRJU1PT9OnTr1y5gncgHUNljRZHR8f379+LxWIdHUw7rL799lsul4sMcra0tPziiy+wrB32zamr2tra06dPR0dHAwBSU1Nra2udnJwAAKtXrw4ODmYymXgHqGZqamrYbHZKSorKvlwmEAio5KbXr19v3rwZjYgUg6TpYcOG/f777/8eVKxUsDWnTlgsVnR0NIPBWLhwYXZ2tlAoRHY8UJ3xouqoubk5MDAwKiqKwWDgHYssHA4H6ZvrpvHjx9fW1hYXFzs44DDN3sjIKDQ0tLGxsb6+/v79+3PmzMGgUpjmVF1LS8v58+clEsnatWsLCwuJROLo0aMBAGPHjh07dize0WmCu3fvnj17VsVzHLoCAwPxDQBZ9eDt27ehoaFLly5VdnXwoVUViUSiEydOIAMaampqGAzGjBkzkAb/8uXLFZq8Dclw9OhRAICfn5+lpSXesXSOSqWiuPtiQkJCTk4OWqV1zdatW5HdPJQ9PwymORUSGhq6cuVKPp8vFAr19PS++uorAEDPnj0XL15sb2+Pd3Sa5sCBA/369cM7CgVwuVwUx0W4urr+8MMPaJXWZcjf7Pr6+m+//VZ5tcCHVpxFRkY+fvz44MGDurq6AoFg1apVyKSi9u34INRVVVVZWVnNnz9fXRbCQ9jY2KA4eNvCwiI8PLylpUUVZilMmTLF1NSUz+c3NTUpY/ATbM3h4MGDB8jkGOSF6dy5c5Ff31WrVuEyM1yrZGVl/fHHHwAA9cpxAICKigqRSIRigcbGxqozbNbDw4NEIuXn558+fRr1wmGaw0hFRcXhw4eRPojc3NwRI0aYmJgAAIKCglRzMKqmSk5O3r17N95RqAQdHZ1ffvkFmeqnIry8vIRCYUVFBbrFwjSnRM3NzdHR0bdu3QIAvHz50szMbMCAAQCAFStWTJo0CePxmRCy1mNQUBDegXSRnZ0d6jOOV6xY8ejRI3TL7KbVq1cbGRllZGSgWCbsm0Nfenq6QCDw9PS8fv16eXk58vIeDm3DV1xcnEAgwDuKbikrK0P3oRV5wfXTTz+hW2b30el0FxeXIUOGPHnyRE9Pr/sFwgYFOoRC4atXrwAAly9fDgkJQQZ8L1y4cPPmzbgMwoQ+QyKRfH198Y5CFdXV1RUXF+MdxedIJNLz58/RGvIC01y3sNlsZAGGESNGZGZmIitbnDp1CuMpe5AMhYWFRUVFX375Jd6BdBeNRkNx3Fw7ExMTPz8/1IvtPgKB4OrqGhcX19zc3M2iYJrruu+//37atGl8Pp9Go6Wmpi5atAhZCx/vuKCPrl69GhER4ejoiHcgKOBwOMp4MUogEH799dfc3FzUS0bFtGnTFi1aVF5e3p1C4EJMiklKSrpy5cq+ffuoVGpaWpoKbkkJteNyuc3NzWoxw0EeqCzEpJ1ga04u6enp2dnZyAadAQEBSNcbzHGqrK2tLT09XWNynLLt3btXKBTiHYVUQqEwNDS0y7fDNCdLa2srMvsvJCTE2NgYALBu3Tpvb2+844I6p3kvf6ysrJS3hH1TU5Oq7SH9KSKROGzYsMWLF3ftdvjQ2rGWlpY9e/aQyeSdO3dyuVxUVjSEMFNSUkKhUDSsKafUh9aampq6ujpnZ2dlFI4WoVAoFou7sMUa7C//XFJS0pgxY8rLy8ePH48snwBznNqBKx0oysLCwsLCAu8oOkEkEjMyMqytrRUNFT60/sOmTZsePnwIAHB2dkZyHKR2vv76a5V9b6jKDh48WF1djXcUnRgwYMD06dMVvQumOYAMkb9+/ToAYMuWLdu2bcM7HKjrnjx5MnLkSPVaYUlO+vr6Sp0gSCKRnj59qrzyUUEkEmNjY5H3gQrcpbR41EZcXFxaWhqyOj4ynR5SX8OHDx8+fDjeUShFS0sLKoukSxMYGNjQ0KC88tHCZDIV3epEq1tzyMZXo0aN+umnn+C+pRqgqKjo/v37eEehrszMzHr37o13FHLJzMxUaAkG7U1zq1evRv7sGxgY4B0LhI69e/eqwiKR6mvZsmV8Ph/vKDrn6upqa2ublZUl5/XamOaQ8UEnT54cN24c3rFAqGGz2fPmzYNjtrtDLBary9ubzZs3Dxw4UM6LtS7NHTx4kEwm4x0FhD4GgzFmzBi8o1AiZaw395n9+/er0fzfhw8fNjY2ynOl1qU5b29vOI1BIx0+fLiyshLvKJRIGevNfcbU1FSNnvpZLNaxY8fkuVKL0lxqamp5ebmHhwfegUDoa2hoiIuLs7a2xjsQ9Zaenv7rr7/iHYW8Zs2aJec4YW1Jc3fv3o2NjbW1tcU7EEhZzpw5g3cIas/U1DQ9PR3vKBSwcuVKeS7TljRnamqqRn+mIEUZGxvDCV7d5+Dg8Msvv+AdhQLy8/OvXbvW6WXakubgxoCabevWrfIPL1BTGLyCAAD06dNH2VWgyNHRUZ7mi1akuePHj8NRo5rt+fPnNjY2eEehXBi8ggAA7Nixo6ioSNm1oIVIJIaEhNTV1cm+TCvSXHZ2Np1OxzsKSFmEQuGRI0fgRD1UtLa2lpSU4B2FAtzc3ExNTWVfo8lzWgcNGtS+RcizZ8+QD3Z2dvI8zENqhEgkqvhCaWpk48aN6jWwtLCw8NKlS1u3bpVxjSa35uzt7Qn/pK+vv3z5crzjglCWmZn5888/4x2FhmAymchC2erCycnpypUrspcH1uQ0N3HixM+OWFtbw22hNU9NTQ2bzcY7CqUzMDBQ6kJMiHv37l24cEHZtaArNja2ra1NxgWa/NAaEBBw586dsrIy5J9kMjkgIADvoCD0eXp6urm54R2F0jU3Nyt1ISaESCR6/fq1smtBV6dvnzS5NWdsbPxpg87Ozm7mzJm4RgQpBYPBMDMzwzsKDTF8+PBVq1bhHYVibt++ffLkSRkXaHKaAwD4+/v36NEDWRl17ty5eIcDKUViYmJkZCTeUWgIBoOhRrP3ERYWFrInb2h4mjMxMfHx8UFeR8yZMwfvcCClqK+vr62txTsKDcFisb777ju8o1CMm5vbTz/9JOOCzvvmqvncInZTg0BWD58qs5g0xqz83YCRI29Vq9NooE/RdIl2VH0nOlz+s2OzZs3CYNysliCRSGo3n0RHR0d291wnaW5X7rN37CZDPRKdqId2bNhxW+DXAkAyqwLvQLqIpqv3rrXRSI+81sm1N8MI73BUhZ+fX2FhIfJb3t43b2dnFxsbi3doSoHNZC8DA4OjR48quxbUbdiwYf369dLW5pCa5iQSybqsBy76Jl9a9FBmeJBcvgR2XJFwb1765r5DetMN8Q5HJSxYsGD//v3ISIL2kRb/HkWkMbCZ7IVsEohBLeji8/mlpaXS0pzUvrntOU9dDc2+MIQvsFQFVZe4wqH/D5kPOCIh3rGoBF9f388WmLO3tw8MDMQvIg2xdetWLpeLdxSK2b17t4xBRR2nubfsxlaRsL8+nCSociYz7cPL3uIdhapYuHBh+2xlHR2dCRMmwJmt3ffy5cuWlha8o1CMkZERjUaTdrbjNFfU2kzWUXovANQFpnqU7OZO1mPQHjNnzmxv0Dk5OcExQ6gIDg5mMBh4R6GYa9euyRg613Gaa+DzjPXUafqu9jAikXli+ND6UWBgIJ1O19XVHTduXKcrVUDyGDFihIyWkWoik8nl5eXSznac5oQSsQjImgoL4UUskbAFAryjUCFIg87BwWH27Nl4x6Jc1tbWGLxpBQAcOnSookLNhiVMmDBh8+bN0s5q8pxWSNW0CAUprIpaPrcHVT+lrrKM08Jq4w4wNOvLMHrdUpfb3NC1z5SV/u91JIdq3vXlsLpTzuuWuuo2Xl+G0TBji7ctjeYU2kQLOwMiCe9v2weVlZXYvGl99erVhAkT1GuZUiKRKONBm9DhAiYXSnPLea1jzdTp69QSDYK2C6W5EUPVbNhEVnOdLoFwqij7LbtR9P+/cmIACAQAJIAAgAQAoAKfJZ884OgQCC76Jot69OWJRMNNmDh+9xDLly9fs2aNu7u7sisqLCy0tLRUr5VoS0pK9uzZc+rUqQ7PwtYcpFxskWBT1iOuSFjJa/3slA4A7V0jBNX4TPgkPLFEkt1c95/sx6YkyvHCzBD3cXRdrfj/4uTkhHcICiORSDJ26dXwOa0Qvh7UVW7KflTQ2vTvHKdG6vi8mjbud5nJV6sKtaHHOjQ09N27d3hHoRhLS8tz585JOwvTHKQsqzLu781LL2A34R0IOso47JDCrK/S43liDZ8/++LFCxaLhXcUitHR0ZGxGBdMcxD6BBLxj9mPilqbBcpfBhJLYgCqeZz/ZD/GZSKKjY0NNm9aZ82aZWdnh0FFKBIKhX5+ftLOwjQHoS+yPD+jSc2aA/LLaak/8O6lGPMRVxUVFdi8afXx8VGv16zIm9aSkhJpqyvDNAeh7D+vH4eV5uIdhXI9YFV+m5GMdxTKcuXKleLiYryjUNiNGzfad/L7DExzEJpeNrLyWhrxjgILZdyW5w01eEehFPfv35fx1lJlMZlMmOYgLBRxmlpFWjFJQyAWpzVW4x2FUmzatMnV1RXvKBQWGBgobYM3rRgHBGHjTk3phVItWj3lRlURTyRa30vTNhVDtk9ROzU1NdL6LjW5NVddVpLz4hneUWiRsNJcVW7KJU1dVHDmIooFCiWSJFYFZl8yNqsHAwB+/fXXjIwMDCpCV0REhL6+foenNDbNpSbc3OA/MS05Ae9AtEWrSGBJVt1lLbjvawRNLXRHlMdJmJIodF2M9g/AbPXgyspKDoeDQUXoYjKZ0rbr1tg0x23V/G3YVQoBEHJb6vGOQqqWvCIAAMPRHt1iy7nsEq6arUDZqa1bt6rj9t5fffVVQ0NDh6fQ7JtLS064dfFsSX6Ojq5er/4D/b/5waGPCwAg43FyzOmj5e/ySFTqwGHe89ZuMrWwAgAc+s+a9JTECXMWvElPra4o7ec2dPOR0BU+Q7mtLTOXfPPw72sNdTWzlwXNWvINACA18faN8yGVxQUUBsPde2zgtxsMjE2k1VtekH9m73YAwJ2osDtRYRY2dgej4zuN/9XjlGtn/yjOz2XoG3pPnnEn6sKcFWunLViGxLn5SOiAoV4AgBcP7x/c9I2nz+Sg3YeQcYk3wk4lx8U0smpMzJkjp/pO/2olkUgEAHz2tUxbtOJO5HkAwLEbD6l0OgCA08r+doo3kUg8nShrl0m1sCv3mbCjZSBQIeYLii9erbqdxKuuJZuZ2M+bZec7GQDwKnifnqG+oXPv4oirvGqWfm/HgTt+oFpbAgAkIlFp1I3y63d51bUGLn0YDnYEIpFmZy1HbYrZmPXo8rBJqBeLIysrK7xD6Irq6mql983djjx/eHNQXuYLpp2jOdM6M/VhS2MDACAt+e6BjatL8nJ6u7obGJukJtzcvXohh93cfmN8zF/G5paDRo4fP/vjKv43wk71dR/i7O4xcuospPBj29ZVlhY5uQykUukpcTG7v1nAbW2VVq+5tY2j8wAAALOHg6fPZHfvsZ3G//jOjf9tWJmfnWFibmFoahZ34U9BG6/TuyQSydHgdTF/HmnjcXv2/4LT2hLz55GQ3f9Y96r9axnvG+AxbnIbl/s08SZyKi05QchvGzrmSwW/2aqohKOsRo1YIHixYWfR+WjzEcP6b1lrOKBf7oGQloISAAC/qfl9fErFzUTHr/zs581sev226EI0clf27sN5J8KM3fq7bFlLpFLKr92m97DWIaLft0UAoEnIR71YHKlp31xYWJixsXGHp9BpzTWyaiOPHyAQCP/5/QzS5KkoLrBx6AkA+OvIbxKJZM1P//P0mSISiQ5sXJWZ+jDxSuT0r1Yg97Y3iz719Q//bc96TXWsyOMHKDT67tBoK3tHiURyYuePj+/cSLpxebjPVGn1jpvpfyYn+wvPUYvWb+00foGAf+H3XwEAgUGbpi1YBgCIOPrbzYjQTm9MT0lMT0mw7+Oy/WQ4mUrjtLK3L/V7cjdu6oKlSEv2s69lzMy5KX9fSb4RM2bGXABAanwcAGDk5JkKfr9VjhhIJlna/6WcTSqKLsQ0vMz+4tctFiM9AABUa+b7u8lttXX6Pe1FHK5+b8fBR35G8tf7xIfcqmoAwPuEB+8THvT5bpm9/3QAgOWY4UlTFtGdUH5iRYw1tzFUmTXpUKGmfXOWlpbSTqGT5jKfPRII+K6eI5BcAwBAck11WUltZbmBkbHH+MkAAF1d3ZFTfDNTH+a+ej4dfExz/y7Q45ODr54+FAj4RuYW92OjkCNIv1vBmyy6gVGH9SqqKPd1S0O9KdN66vylyBEylSrPjS8e3AMAUGi0mD8/bG1JJlMBAIVvstrT3KdfS5+B7raOvfKzMyqKC/SNjLOfPzG1sHIe7NGFmFWKDiBwlPPCUSKRlF25RWFa6Pd0aGPVs4tK34WEk4wNjVydJWJxa2mFvf/09jaamMvTM9AHAJTF3KRaW9rNnvKhEKFIxGtjOCplnEQJp6VNLMJg7xQmk4nNm9Z58+Y5OjpiUBG6tm/fvn79+g4bdOikuSZWLQDAwubzX6PmpgYAgIGpefvoZH0jYwBAa9PHVSsotA7W/KTQPi7phxReW1l+8+LZT68hkSnS6lVUc30dAMDS1l7aKGppGutqAABvM9LeZqR9elyPRGn//OnXAgAYPd3vryN7k2/EWFjbiUUir0nTFa1UBTUJ+fHVZcoomVdVI2hsEtOoD/1XIUeM3QcMOryLSKNyKqrEvDa6w4eXp0IOt62ugdbDRiwUNuXkMyeMak9/rSXlQCJRUporam0u5rT0Vf424VVVVdLmbKLr4sWL8+bNU7seuqdPn0rrm0MnzdH0DQAADbWfz30xMDQGADQ3fNyJqqG2FgDAMOr4ftIkogAAHiFJREFUEbrjwhn6AABPnylBuw9+dirxamSH9baTyPdrQaMzAADNdbVSzhMAAB3+hiGxLflx53jfAHkqAgCMmDwz6sTBB7euMW16IP+U80ZVpk/UU9JGPGKhEADQd+1SY7f+gmY2xcqCbPIhobCLygAA7cmLXVSK/FPYwpYIhWTTj79jDRmvAQAMJ6WkOZou0eyTv2rKI5FIsPmL2Lt3bwMDAwwqQtfZs2el9c2h8wqin/sQAEDG46S8rJfIkaK3r/ltPAvbHqYWVs31dekpiUgX2L3YSABA/8GeChQ+aCgAIP3BvYI3We2Ft3E5MuoFAFDp+gCAqtIiJEMJhbL+Ezr0c9HTI5UXvXt67zZyRCj42KlsYGICACjKzUbeqz77/2sAAP3chgEA7kSeb274MJYi71Un70z1jYwHjx7f0lCfn53h6Ny/a0/ZqkYHEDb1GayMkimWZoBAaM4rpNlZG/bv057jAACtRaVAR4fW48NaGq2FpQAAuoMdUZ9B0NXhVFQhx0W8ttKYv3XIJOQNLOomWtqbYpLmMPPdd98NGDAA7ygUJmNLIHRaczYOPUdNm5MSF/Pz6gU2Tr0JBEJ5Qd7iTTvGzQqYu3rdyV3/ObptXa8Bbqz3layqCkvbHkgHvPyFj5w868GtaztXBPTo7SwUCiqL3s1b++PkwMUy6nVyGaCjq5v17NHmhTO47JatR89Z2kntgaYxDCbN+/pG2J9Hg9fF2Dvpkcgl+TntZ109Rty/FhUd8vuLlHt1Ne8bWR8bjyOnzIyPDq8oLvjBz8fWsXdzQ31NZdnuczGOffvL+Io8faamJtwCAIycPEv+74OKc6Qp5e+/LplsMWZ4xfU7BF0dg3692AUlhgP6Wo4ejrTmqNaWuuQP3f/solKCrg69h7UOkWjqMag2JbXwfBTdzqYk6npbDYvRy5EgZexoN1Ew6S/DsjWXn59vYWFhaGiIQV0oWrJkyaFDh4yMOug9QO0Hv2zzroBvNpjb2FUWF9RVV/Ub5GHr1Bt5KAvafcjGode77AwOm+01cXrwHxeoCu6msTx4z9zV68ytbUvf5dZVVfYbNMy+Vz/Z9VpY2y3fstvU0qqqpFAiluhROtl21n/1D4HfbrCwtquuKG1qqLO2/7gc/tAxX85ZvtbYzLK0IM/GoVf7O2IAAJlKCz5xYexMfxKFWpiTxeNxPH2m0PU7+Q/vMsQTAKBLJHr6TFXo+6DKMptYdKJS5gO4bPqG6TOyOvFh7oGQxldvaLYfxr61FpUyHD7OamAXlVGtmTp6egAAl/+sMfMaWhJxLff306bD3KnWTCV1zOnp6LQIMZrshVmaO3LkyOvXrzGoCF2VlZXSHtrgzl4dizl99OqZ4+3jS9CVl/Vy18p57iPGbvjfCUXvVdmdveoFbUvTE3BZVhdH1hT6LhePHtSOp1Kia+3atatWrcLgcfLIkSPjxo1Tu+fWyspKS0vLDp9btWWFkrysl1fPHJN2dvGmnyxtsFgVujQ/98DGb+pqqoh6esjsDo1hokfe6eyxKfuRjGuydh1iPU7793GyhWlbTd2/j+sZ6I+IOolikM+/3cIuLEUxgK/tnbHJcQCAxsZGaXM20fXdd99hUAvqrK2lTnHRljTXXM/Keir1fyC3FaNpiXx+W1sbd9DIcbMWf+vkomZ/LTvVm2E0xNgiTfpik33XLum1YsG/j4sFAuRh8zOo96YN3LVJIuigvdm1AHpQ9XvRsevAEggEeh0FiTo17ZtbvHjx4cOHO+yb05Y0N2T0hPAnCqzcPWf52jnL16IeRq/+X5y8nYp6sSqCpks00ZP1zpFkrPTBZbJRzExQLM2MTLWjSt3pHXVCoRCbNHfkyJF58+Z5eXlhUBeKysvLO+yC0+QVSiBc/NDbfaixUsZtqJrhJsyfnIdhWaNAIECWhFC2yZMnq93OXgCA8+fPS2uBwjQHoUkHgD0unmPNbfEORLncDM039RlMUf4Er09h9tA6ZcoUdUxzNjY2WrfeHISjMWY2DOUMLlEFdF3iJMseDF2sO3yEQiE2rbnLly+/e/cOg4rQtWDBAmkrDsA0B6FvuAlze79h3qZqNilSHm6GZssc+o/Do7kqEoloNCzWZ3769Gl5eTkGFaGrrKxMWt+ctryCgDDmZmjWh2FkpPf67/fqt+OnNH0ZxnsHeOkAHJZaEIvFzc3NVPkWzummSZMmqeOuNxEREdL+DMA0BykLTZe4xsm1ns97z+MUcZrluEN1OdIN6brE7f2G4ZLjAABsNpvBwOitro+PDzYVocvWVmoTG6Y5SImIBMJOZ48yLru2jft7QUZtG08owWIpIbRIADDUI33jONDDxJKB1dY2HcIyzRUXF9PpdHNzc2yqQ4uvr+/Vq1c7PAXTHKR0dlSGHZXxS3+v+7XlJB1dtpD/gFXJl4jMSFQ9gk4Fj80XS2wpdD0dFfisq1vJZYuBZISpjbO+0bvW5uEmTFcDU7y/hZimuZiYGCsrq/nz52NTHSqEQmFlZaW0szDNQRixodAX2vVFPk9hOlTzOL0ZRjRdYmZTHUckcDU0U5HPb1rqKbpER5oBAYDxKtOgaWlpkbYJKeqGDRuGzStdFOnq6sbExEg7C6fuqxmVnboPKVViYmJKSsrOnTvxDkQtdTygxIBI0lX7hbs1k1gitutoWXlIs1VWVkpbGhd1ubm5ycnJ2NSFlqampsWLF0s723Gas6Xpl3Lgds6qqILbqq9Z+0hB8qiqqsJsc4b3799fv34dm7rQwuVyWSyWtLMdP4G7GZrxxWK+WEzCZOEXSH4VvNYvLdVvTBPUTVVVVZ6eCmwt0B39+vVr+mRTKrVgZmZ2+vRpaWc7zmK6BML6Xm6RFfnKDAxS2J2aUhsqw9tEA2cXQLJh2ZpjMpkzZ6rZTkxEIpHJZEo9K+2Es77xul5u616l+FjYmZAoGjxFUR1IKritTQK+GYm60kHWLhOQpqqpqcEszQmFwtjY2Dlz5mBTHSrevn0bGhq6b9++Ds/Kem3cm2541XNqVEV+Tkt9Hb9NaREqXU1NjbGxMTarOyiDHZVuqEeeaNljmHascQR9pqamhkwmYzZujkgkHj9+3MfHR41W1mxsbGxpkbo4biejY0g6Ou1jndTX/Pnzf9yxo29ftf9CIO305s2b/v0xbcVPnjy5tbVVjdKcq6vrrl27pJ1Vs0GAEKSFsrOzMU5zmzZtwrK67qNSqTLWNYAvUiFI1b1588bFxQXLGl+9evX27Vssa+ym27dvh4SESDurFWnO0dFR2nbcEKT6sH9oLSoqioqKwrLGbqqqqhIIpG6YqxUPrSUlJdL2qYUgFVdeXu7u7o7Z+weEp6cnNpslomXOnDkyAtaKNGdlZSUWq9P6PxDULikpyd7eHuNKmUzmjBkzMK60OwwMDGScVaeE3WVcLlftRnVDECIhIWH8+PHY13vu3Dk2W21mfO7du/fRI6kbMWtFmnNycuJyuXhHAUEKq62tff/+/cCBA7GvOisrKy0tDft6u6aoqIhMJks7qxVpzsDAIC8vD+8oIEhhiYmJuDTlkE3sMVsTpft+/vlnNzc3aWe1Is316tWrtbUV7yggSGE4prmBAwd+8cUXuFTdBebm5jKWAtWKNDdgwID4+Hi8o4Agxbx//14kEg0aNAiX2oVCoYw1P1TN1KlTZZzVijRnZmbGYDCKizVnJz1IG5w+fRrH151EIjEpKSk3NxevAOTHYrFEIpGMC7QizQEAJk6c+OLFC7yjgCB5NTQ0JCUlzZo1C8cYgoKC1GLAqZGRUUREhIwLOt4LQvMUFxdv3LgxOjoa70AgSC4HDhxQu+21VJa2tOYcHByMjIxevnyJdyAQ1Dkej3flyhVVyHFbt27FO4TOxcXFHT16VMYF2pLmAABLliy5dOkS3lFAUOcuXry4bt06vKMAyP6wMobdqoji4mLZk+G05aEVERQUtHDhQszW1IegLsjKyjpw4MC5c+fwDgQgvfs8Hs/W1hbvQGThcDhEIpFEkroVlHalOdhDB6m+SZMmXbhwwdxcZbbCVn9a9NCK9NBNmjTpxIkTeAcCQR377bfflixZolI5Ljg4OCsrC+8oZJkzZ059fb2MC7QrzQEAli9fnpOTo/rdDZAWSktLKygoCAgIwDuQfxg8eLAqb9vK4XBqampMTExkXKNdD63tRo8eHRcXp6+vj3cgEPSBRCLx9/e/dOmSCi4By+fzZfR84UsikQiFQtkbWmlpmgMAeHt7wzYdpDrGjBkTFxeH8fKZcuLxeEQiUcakURWndQ+t7RITE+ErV0hFzJgx46+//lLNHIfsubNmzRq8o+jYyZMnz549K/sa7U1zFArl4cOHQ4cOra2txTsWSKstXLhw3759NjY2eAci1ZAhQ8hkcmVlJd6BdODdu3cODg6yr9Heh1aEWCz29fVdu3atj48P3rFA2igoKGjdunW9evXCOxBNpr2tOYSOjk5sbGxCQsL+/fvxjgXSLmw2e9KkSRs2bFCXHJeQkCB7IRBcyBOStqc5xN69e21sbFasWFFXV4d3LJBWyMrKmjZtWnh4uKOjI96xyKu4uPjPP//EO4p/ePXq1YoVKzq9DKa5D+bNm7du3bp58+aFh4fjHQuk4a5fv37gwIGkpCQzMzO8Y1HAsmXLjIyM8I7iHwoKClxdXTu9TNv75v7t0KFD6enpP//8c6f9mhDUBSdPnqyurt6xYwfegWgR2Jr73Pr167dt27Zx40bYrIPQlZeXN23aNEtLS7XOcUuWLME7hI9aWlpg31wX9evXLzo6Wk9Pb+TIkbGxsXiHA2mC06dP79ix4/Tp076+vnjH0i2jR48+duwY3lEAZP/lyZMnyzNpBKY5qQICAu7cufPq1avAwEA12rASUjV1dXWLFi0SCoUXL15kMpl4h9NdixcvDgoKwjsKgLSOx4wZI8+VsG+uc/n5+fv37+/Vq5evr6+6vPuHVERISEhOTs7KlStdXFzwjgU1LBaLzWarUec1bM11rnfv3iEhIV5eXsHBwVu2bCkrK8M7IkgNJCUlffnllwQC4fDhw5qU45Ct8rZt25aTk4NvGOXl5XLuvwxbc4q5e/fuH3/84ebmtmbNGpVaFAxSHRUVFb/++iuFQtmyZYupqSne4ShFTU3N48eP8d14zNvbOzExkUKhdHolTHNdERcXd/PmTQsLiyVLltjb2+MdDqQq2trazpw5c/v27S1btgwfPhzvcDRZcXFxRESEnDvywDTXdXFxcaGhoT179ly6dKmzszPe4UA4O3nyZFhY2Pr16+fOnYt3LBhZtWpVSEgI3lF0Dqa57rp3715oaKiFhUVgYOCwYcPwDgfCQXh4+LFjx5YtWybPxCNNcvPmzRcvXmzbtg37qnNzc5lMppyzMmCaQ8fTp0/PnTvX1NS0cOHCKVOm4B0OhJGLFy8mJyf37ds3KChI9gK2ELq8vLzu379PJpPluRimOTS9ffs2PDw8NTX166+/DggIgL/3mkosFp89e/bs2bOzZs1avny5qs30xBKfz3/8+LGc49fQUlxcHBsb+/3338t5PUxz6Kuvr7906VJYWNiMGTMCAgJ69uyJd0QQapqamsLCwsLCwpYsWbJ06VJ5XvNpvMjIyJKSkh9//BHvQKSCaU6JYmJiIiMjTUxMFi1a5O3tjXc4ULfk5uZGREQ8e/YsMDBw8eLFeIejWp4/f96rVy9jY2NsqktPT+/Zs6f8jWiY5pTu+fPnSUlJ9+/fDwgICAgIgH//1U5KSkp4eHhra+v8+fOnTp2KdzjajsPhTJw48cGDB/LfAtMcRqqrqyMjI6OioubNmzd+/Ph+/fp9enbixImmpqYRERH4BQh9rq2tLSoqKjIy0t3dfdasWYMHD8Y7IpUWFxf3/PnznTt3Krui3Nzc3NxchUYmwzSHtTt37ly4cIFIJPr7+7e/kx0yZAgAwMPD4/jx43gHCIG8vLyoqKibN2/6+/sHBgZqwHx7bFy+fNnFxaV///54B/I5mObwkZWVFRUVlZKSMnfu3Js3b9bU1AAAiETi1KlT//vf/+Idnfa6ffv25cuXhUKhr68vvjOZIGnOnz/v5+dHp9Plv0Vd95dVdwMHDhw4cCCbzb58+XJ1dTWBQAAACIXC+Ph4CwuLVatW4R2gdqmrq7t8+XJkZKSXl9fatWvd3NzwjkhdFRQUREVFbdmyRUnl5+bmxsfHf/311wrdBVtz+EOeWNvp6+uvW7du5syZ+EWkRdLS0qKiojIyMvz9/f39/Q0MDPCOSO1du3aNz+f7+/sro/B3794RCARFB2nBNIezsYF+nLa2T4+IxWIjI6MVK1bANoVSxcfHx8fHGxkZTZgwYejQoR+OEghWZJo+EY7r1igwzeFDJJH8lvciiVUuKSonGOpLJBLkuRUAQCAQCASClZUV3jFquLq6OgMDg89mqhjpkQtbmxxoBnNtenqbWuMXndo7derUsmXL5FnBXH7Nzc2HDx/evn27ojfCvjkctAgFS9ITZlv3/MnZg+DsUVJSQqFQyGQylUqVc44ehALpa9+yRcKL5e84IuEEix6YhqRBvL29lyxZEhYWhmKZd+7cIZFIXbgRtuZwMOfpzW+dBtJ14ZORSosoz5tt3XOMmQ3egagrNpstFApRnPCbkZHRo0cPExMTRW+EaQ5rEeX5dXzuUCMLvAOBOiGUiKMq3h1xHYV3IGqsqKjI0NCwC4kJXXAvCKxlNtUaErvS8IYwRiTosNq4FTy5dhuAOuTo6Lhs2bLS0tLuF5WamtrlR2CY5rAmAsCcRMU7CkguvehGFdwWvKNQb5cvX0YlzUVGRnZ5LzH4CgJr77mtYgA7CtRDk4gvhj+r7iESiYMHD66srLS27tab6w0bNtja2nbtXtiagyBIuahUakpKyv/+978ulyAUCs3MzLp8O0xzEAQpXWBg4OzZs6urq7t2++bNm1NTU7tcO0xzEARhoWfPnrq6unw+X9EbhUJhcXFxd9Zhh2kOgiCMmJmZTZkypbGxUaG7iERidHR0d+qFaQ6CIOzExcVdunRJoVuQYcbdqRSmOQiCsEOhUFavXi3/9Ww2e9q0aURit8aEwDQHQRDWUlJS9u/fL8+Vjx49WrBgQTerg+PmIAjC2qhRo3R1dVNSUkaN6mQu3cSJE7tfHWzNQRCEA29vb09PT9mdbhwOJy4urvt1wTQHQRA+SCTS3r17X7x4Ie2CW7dusVis7lcE0xwEQbjZtm0bj8fjcrkdnnVycvLz8+t+LTDNQfJiNzWmJd/FOwpI03h5eUmbHeHu7s5gMLpfBUxzkFzqqivXTh915cwfeAcCaSASiTR9+vTPDl66dOnq1auolA/THCQXIV8gECg8TQeC5GFtbX3+/Pm0tLRPD4aHh3t6eqJSPhxQogaEQuGNsFPJcTGNrBoTc+bIqb7Tv1pJJBJP/bw15e8ro6bNWRm8BwDw7P6dI1u/t7J3/PlsDJlKk3YXAEAkEt26dO7h31ffV5TpGxi5Dh8Z8M0PBsYmK3yGcltbzj3IRi6LOLLv5sWzSzb95DZi9Ab/iQCA0vzchcP7AQB+j71vamElowrZOOzm6JAjz5LuctjNPV1c6fqGWc8enrn3sjjvzbavZ/d2HbQjJAK58r9L5hTlvt538W8bh54AgKK3r6NOHMzLfEEg6PRxdZ+7er1j3/4AgFuXzv31+97Bo3w47OaCN5kUCnXktNl/h5/xXbZmzvK1SFFRJw5eDzu1bu+xIaN9lPwTgxRmYmJCJpPr6+uRpYZFIlFUVBSNRkOlcNiaU3USieRo8LqYP4+08bg9+3/BaW2J+fNIyO7NAICF6zabWlilxMW8fv6krqbq9K/b9fRIQbsPkqk0GXdJJJIjW7+/dOx/NVXljn1d9EikZ4m3AUFWDGQy1c1rNACAxjDw9Jns6TOZTKbKqEI2Po+759uv70aHc1tb7Jz6VBTmpyXHy/OtyM/O2LVqQdbTR9YOPZl2DpmpD3evXlCSn9N+QXpKQktDvef4KWNmzPWZPZ9AIKTEXRGLxcjZ1IRbAIA+XwySpy4Ie3Q6/fLly6dOnUK28eza7jYdgq05VZeekpiekmDfx2X7yXAylcZpZW9f6vfkbtzUBUsd+rgs27r7t3XLz+zbbmxuyWlp+mrDNvvezrLvYlVVpKckmJgzt5/6y4xpAwCoKC4wMJK1Wj/D0GjRuq0Zj5PNrKyDdh9CDqYlJ8gITEZpidciS/JzLO3sg4+HmZhb8jic5ePlSj3nftspaOOt2XVg+ISpAIB71yJD9+24cvrY+n3HkQvMrW13hV4mUT4szjxgqFfWs0fZzx+7eox49/pVTWVZX7chsr9SCF+rVq3KzMwsLy9ftGhRbGwsWruDwzSn6l48uAcAoNBoMX8eRY6QyVQAQOGbLIc+Lq4eI8bO9L8fG1VTUTZo5Lgv/RZ2ete77FcAgAl+C5AcBwBAngfRDUzGjZlPHgAApi9aYWJuiZQgT3Ws9xUl+Tm6RGJRTnZRTjYAgM/nAQAK3mS2X+PuPbY9xwEARs+Ym/XsUUpcjKvHiCd3/wYAjJg0owtfKYQlV1fXjIyMiRMnopXjYJpTA411NQCAtxlpbzP+0UGrR6IgHybMmX8/NgoAMHneEnnuQk5Z2NopOzBpmhvqAQBWPRRb17+xjgUAEAmFNy+e/fQ46ZPqqP/MmINHjTMwMk5PTmxuqH967xaRRPYYP0mhSiFcuLm5ubm5oVggTHOqjsbQBwAs+XHneN+Af58Vi8XnD+xGPofu2747NAZpHMm469WTZABAI6vm36URdHQAABKJWFow7f1cnQYmA4VOb09bn9Eh6CDV/PsUlc4AABiZmR+78UDOivT0SCMmz7p58eyZvf9tZNV6+kymMVBrIEBqBL6CUHX93IYBAO5EnkcaQQCAvFfp7WfjLpx++yrdZYin54SpVSVFZ/Zt7/Qu50EeAIC70X811H7IdHlZL5EPhiYmAADkkbC5sT7r+eP2iih0BgCg7n0Vn8cFAAgEfNmBydDH1R0AcPviOU4rGwAg4Le1nzIwNgUAVJYWIaeK3r6uLClETln1cDQ0NWtk1d6N/gs50lRf9760WHZdo2f4IT2VAADvSTPlCQ/SPLA1p+pGTpkZHx1eUVzwg5+PrWPv5ob6msqy3ediHPv2L8nPuXLmKIlCWb5lN8PQuPBN5pO7cf2+GDJ+dqCMu0ZOnnE3OryiMH9jwEQbh17spsaayrLfLt60dnAaONS7qqTot/XL7Xr2LSvI43E+blFqaGJqYWNXU1G2KWAKVV9/kv8iGVXI/oom+C28dy0qPztj3axxTDv72qqy9lNGZua2PfuUF+T9GDjFxNyy+O3r9vajjo5OwDc/nPp5a9iB3XcvX6DSGZXFBQOGerW/f+iQjUNP+97OJfk5+sYmAz1GdPunAakl2JpTdWQqLfjEhbEz/UkUamFOFo/H8fSZQtc3EPDb/vjpR6FA4L96vYW1HY3OCNp1iKinF374l6LcbGl3AQBIFOq242FjZ/lTaPSS/Bw+n+c9aTqZRgUAzFm51mvidF2iXkVx4ZDR4z/ryVqz66B9H5emBlZDbTXD0FhGFbKZmFv+9+RfX3iNEouF5UX5Vvb/eAEStPtgX7ch3NaWpnrWtIXLbR17tZ8aNXX2d7/87ug8oK6qsqwgn2nr4OoxstPq+g/xBAB4Teju0oyQ+iJIJHAfSkwtSosPtO1t2lk/vVZZOLwfmUo9c+8l6iULBPwjW79/+fD+7rPRjv0GKHr7xYr8eTa9PU2YqAcGYQn+fYNQxuNwft+6VtrZ8b6BQ0ZPwCaSU3uCnybebONyB48a34UcB2kMmOYglIlEgqynj6SddfXs/DETLa3NjSYWTO9JM6cuWIpZpZAKgmkOQhld3zD8Sa5Ctyh6vZxkv52AtAd8BQFBkIaDaQ6CIA0H0xwEQRoOpjkIgjQcTHMQBGk4mOYgCNJwMM1BEKThYJqDIEjDwTQHQZCGg2kOgiANB9Mc1nrQ9GXuogWpEBM9sp6uLt5RQN0F0xzWSASdSh4H7ygguWQ11TnS4Lrqag+mOawNNbFsFLTJcSGEswZhWx+GkYkeGe9AoO6CaQ5rUyzty7ns9MZavAOBOnGhNHelYycLvkNqAa4ejI9N2Y9tKDQrCt2ayiDCvjqVIQGEej6vXsC7UlFwzG2MLYWOd0QQCmCaw83VqsL7teUAgLyWBrxjgT6wotLFEslQI8uv7Z31iXp4hwOhA6Y5CPpIDPtxNBH8mULQR/D/g0aCP1YIgjQcTHMQBGk4mOYgCNJwMM1BEKThYJqDIEjD/R/ruUC12lPWvAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# check the graph workflow\n",
    "from langchain_opentutorial.graphs import visualize_graph\n",
    "\n",
    "visualize_graph(app)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Excution the Agent\n",
    "\n",
    "Execute the agent to interact with an SQL database through the entire process.\n",
    "\n",
    "The agent retrieves information from the database based on user queries, generates and executes SQL queries, and returns the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.runnables import RunnableConfig\n",
    "from langchain_opentutorial.messages import random_uuid, invoke_graph, stream_graph\n",
    "from langchain_core.messages import HumanMessage\n",
    "from langgraph.errors import GraphRecursionError\n",
    "\n",
    "\n",
    "def run_graph(\n",
    "    message: str, recursive_limit: int = 30, node_names=[], stream: bool = False\n",
    "):\n",
    "    # Configure settings (maximum recursion depth, thread ID)\n",
    "    config = RunnableConfig(\n",
    "        recursion_limit=recursive_limit, configurable={\"thread_id\": random_uuid()}\n",
    "    )\n",
    "\n",
    "    # Input the user's query\n",
    "    inputs = {\n",
    "        \"messages\": [HumanMessage(content=message)],\n",
    "    }\n",
    "\n",
    "    try:\n",
    "        if stream:\n",
    "            # Execute the graph with streaming\n",
    "            stream_graph(app, inputs, config, node_names=node_names)\n",
    "        else:\n",
    "            # Execute the graph without streaming\n",
    "            invoke_graph(app, inputs, config, node_names=node_names)\n",
    "        # Retrieve and return the output state values\n",
    "        output = app.get_state(config).values\n",
    "        return output\n",
    "    except GraphRecursionError as recursion_error:\n",
    "        print(f\"GraphRecursionError: {recursion_error}\")\n",
    "        # Return output state values even in case of a recursion error\n",
    "        output = app.get_state(config).values\n",
    "        return output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mfirst_tool_call\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "Tool Calls:\n",
      "  sql_db_list_tables (initial_tool_call_abc123)\n",
      " Call ID: initial_tool_call_abc123\n",
      "  Args:\n",
      "==================================================\n",
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mlist_tables_tool\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "=================================\u001b[1m Tool Message \u001b[0m=================================\n",
      "Name: sql_db_list_tables\n",
      "\n",
      "Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track\n",
      "==================================================\n",
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mmodel_get_schema\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "Tool Calls:\n",
      "  sql_db_schema (call_7LsScKaQEKiBo1xZ3QtEXEmX)\n",
      " Call ID: call_7LsScKaQEKiBo1xZ3QtEXEmX\n",
      "  Args:\n",
      "    table_names: Employee\n",
      "==================================================\n",
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mget_schema_tool\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "=================================\u001b[1m Tool Message \u001b[0m=================================\n",
      "Name: sql_db_schema\n",
      "\n",
      "\n",
      "CREATE TABLE \"Employee\" (\n",
      "\t\"EmployeeId\" INTEGER NOT NULL, \n",
      "\t\"LastName\" NVARCHAR(20) NOT NULL, \n",
      "\t\"FirstName\" NVARCHAR(20) NOT NULL, \n",
      "\t\"Title\" NVARCHAR(30), \n",
      "\t\"ReportsTo\" INTEGER, \n",
      "\t\"BirthDate\" DATETIME, \n",
      "\t\"HireDate\" DATETIME, \n",
      "\t\"Address\" NVARCHAR(70), \n",
      "\t\"City\" NVARCHAR(40), \n",
      "\t\"State\" NVARCHAR(40), \n",
      "\t\"Country\" NVARCHAR(40), \n",
      "\t\"PostalCode\" NVARCHAR(10), \n",
      "\t\"Phone\" NVARCHAR(24), \n",
      "\t\"Fax\" NVARCHAR(24), \n",
      "\t\"Email\" NVARCHAR(60), \n",
      "\tPRIMARY KEY (\"EmployeeId\"), \n",
      "\tFOREIGN KEY(\"ReportsTo\") REFERENCES \"Employee\" (\"EmployeeId\")\n",
      ")\n",
      "\n",
      "/*\n",
      "3 rows from Employee table:\n",
      "EmployeeId\tLastName\tFirstName\tTitle\tReportsTo\tBirthDate\tHireDate\tAddress\tCity\tState\tCountry\tPostalCode\tPhone\tFax\tEmail\n",
      "1\tAdams\tAndrew\tGeneral Manager\tNone\t1962-02-18 00:00:00\t2002-08-14 00:00:00\t11120 Jasper Ave NW\tEdmonton\tAB\tCanada\tT5K 2N1\t+1 (780) 428-9482\t+1 (780) 428-3457\tandrew@chinookcorp.com\n",
      "2\tEdwards\tNancy\tSales Manager\t1\t1958-12-08 00:00:00\t2002-05-01 00:00:00\t825 8 Ave SW\tCalgary\tAB\tCanada\tT2P 2T3\t+1 (403) 262-3443\t+1 (403) 262-3322\tnancy@chinookcorp.com\n",
      "3\tPeacock\tJane\tSales Support Agent\t2\t1973-08-29 00:00:00\t2002-04-01 00:00:00\t1111 6 Ave SW\tCalgary\tAB\tCanada\tT2P 5M5\t+1 (403) 262-3443\t+1 (403) 262-6712\tjane@chinookcorp.com\n",
      "*/\n",
      "==================================================\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "\n",
      "SELECT * FROM Employee WHERE FirstName = 'Andrew' AND LastName = 'Adam';\n",
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mquery_gen\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "\n",
      "SELECT * FROM Employee WHERE FirstName = 'Andrew' AND LastName = 'Adam';\n",
      "==================================================\n",
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mcorrect_query\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "Tool Calls:\n",
      "  db_query_tool (call_1piJSw5DNNlHsSCT5WbR5lLO)\n",
      " Call ID: call_1piJSw5DNNlHsSCT5WbR5lLO\n",
      "  Args:\n",
      "    query: SELECT * FROM Employee WHERE FirstName = 'Andrew' AND LastName = 'Adam';\n",
      "==================================================\n",
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mexecute_query\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "=================================\u001b[1m Tool Message \u001b[0m=================================\n",
      "Name: db_query_tool\n",
      "\n",
      "Error: Query failed. Please rewrite your query and try again.\n",
      "==================================================\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "Tool Calls:\n",
      "  model_check_query (call_RWkZ7mD5QfGTavEEDywReBlU)\n",
      " Call ID: call_RWkZ7mD5QfGTavEEDywReBlU\n",
      "  Args:\n",
      "    state: {'messages': [{'content': \"SELECT * FROM Employee WHERE FirstName = 'Andrew' AND LastName = 'Adam';\", 'type': 'human'}]}\n",
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mquery_gen\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "Tool Calls:\n",
      "  model_check_query (call_RWkZ7mD5QfGTavEEDywReBlU)\n",
      " Call ID: call_RWkZ7mD5QfGTavEEDywReBlU\n",
      "  Args:\n",
      "    state: {'messages': [{'content': \"SELECT * FROM Employee WHERE FirstName = 'Andrew' AND LastName = 'Adam';\", 'type': 'human'}]}\n",
      "=================================\u001b[1m Tool Message \u001b[0m=================================\n",
      "\n",
      "Error: The wrong tool was called: model_check_query. Please fix your mistakes. Remember to only call SubmitFinalAnswer to submit the final answer. Generated queries should be outputted WITHOUT a tool call.\n",
      "==================================================\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "\n",
      "SELECT * FROM Employee WHERE FirstName = 'Andrew' AND LastName = 'Adams';\n",
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mquery_gen\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "\n",
      "SELECT * FROM Employee WHERE FirstName = 'Andrew' AND LastName = 'Adams';\n",
      "==================================================\n",
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mcorrect_query\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "Tool Calls:\n",
      "  db_query_tool (call_F5lf0EVADdpckD5tJXOM9jQ3)\n",
      " Call ID: call_F5lf0EVADdpckD5tJXOM9jQ3\n",
      "  Args:\n",
      "    query: SELECT * FROM Employee WHERE FirstName = 'Andrew' AND LastName = 'Adams';\n",
      "==================================================\n",
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mexecute_query\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "=================================\u001b[1m Tool Message \u001b[0m=================================\n",
      "Name: db_query_tool\n",
      "\n",
      "[(1, 'Adams', 'Andrew', 'General Manager', None, '1962-02-18 00:00:00', '2002-08-14 00:00:00', '11120 Jasper Ave NW', 'Edmonton', 'AB', 'Canada', 'T5K 2N1', '+1 (780) 428-9482', '+1 (780) 428-3457', 'andrew@chinookcorp.com')]\n",
      "==================================================\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "\n",
      "Answer: The personal information about employee Andrew Adam is as follows: \n",
      "- Employee ID: 1\n",
      "- Last Name: Adams\n",
      "- First Name: Andrew\n",
      "- Title: General Manager\n",
      "- Birth Date: February 18, 1962\n",
      "- Hire Date: August 14, 2002\n",
      "- Address: 11120 Jasper Ave NW\n",
      "- City: Edmonton\n",
      "- State: AB\n",
      "- Country: Canada\n",
      "- Postal Code: T5K 2N1\n",
      "- Phone: +1 (780) 428-9482\n",
      "- Fax: +1 (780) 428-3457\n",
      "- Email: andrew@chinookcorp.com\n",
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mquery_gen\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "\n",
      "Answer: The personal information about employee Andrew Adam is as follows: \n",
      "- Employee ID: 1\n",
      "- Last Name: Adams\n",
      "- First Name: Andrew\n",
      "- Title: General Manager\n",
      "- Birth Date: February 18, 1962\n",
      "- Hire Date: August 14, 2002\n",
      "- Address: 11120 Jasper Ave NW\n",
      "- City: Edmonton\n",
      "- State: AB\n",
      "- Country: Canada\n",
      "- Postal Code: T5K 2N1\n",
      "- Phone: +1 (780) 428-9482\n",
      "- Fax: +1 (780) 428-3457\n",
      "- Email: andrew@chinookcorp.com\n",
      "==================================================\n"
     ]
    }
   ],
   "source": [
    "output = run_graph(\n",
    "    \"Let me know all personal information about employee Andrew Adam.\",\n",
    "    stream=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mfirst_tool_call\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "Tool Calls:\n",
      "  sql_db_list_tables (initial_tool_call_abc123)\n",
      " Call ID: initial_tool_call_abc123\n",
      "  Args:\n",
      "==================================================\n",
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mlist_tables_tool\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "=================================\u001b[1m Tool Message \u001b[0m=================================\n",
      "Name: sql_db_list_tables\n",
      "\n",
      "Album, Artist, Customer, Employee, Genre, Invoice, InvoiceLine, MediaType, Playlist, PlaylistTrack, Track\n",
      "==================================================\n",
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mmodel_get_schema\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "Tool Calls:\n",
      "  sql_db_schema (call_xLI2mKHH0Bx5RfFI2ASXQqJ5)\n",
      " Call ID: call_xLI2mKHH0Bx5RfFI2ASXQqJ5\n",
      "  Args:\n",
      "    table_names: Customer, Invoice\n",
      "==================================================\n",
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mget_schema_tool\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "=================================\u001b[1m Tool Message \u001b[0m=================================\n",
      "Name: sql_db_schema\n",
      "\n",
      "\n",
      "CREATE TABLE \"Customer\" (\n",
      "\t\"CustomerId\" INTEGER NOT NULL, \n",
      "\t\"FirstName\" NVARCHAR(40) NOT NULL, \n",
      "\t\"LastName\" NVARCHAR(20) NOT NULL, \n",
      "\t\"Company\" NVARCHAR(80), \n",
      "\t\"Address\" NVARCHAR(70), \n",
      "\t\"City\" NVARCHAR(40), \n",
      "\t\"State\" NVARCHAR(40), \n",
      "\t\"Country\" NVARCHAR(40), \n",
      "\t\"PostalCode\" NVARCHAR(10), \n",
      "\t\"Phone\" NVARCHAR(24), \n",
      "\t\"Fax\" NVARCHAR(24), \n",
      "\t\"Email\" NVARCHAR(60) NOT NULL, \n",
      "\t\"SupportRepId\" INTEGER, \n",
      "\tPRIMARY KEY (\"CustomerId\"), \n",
      "\tFOREIGN KEY(\"SupportRepId\") REFERENCES \"Employee\" (\"EmployeeId\")\n",
      ")\n",
      "\n",
      "/*\n",
      "3 rows from Customer table:\n",
      "CustomerId\tFirstName\tLastName\tCompany\tAddress\tCity\tState\tCountry\tPostalCode\tPhone\tFax\tEmail\tSupportRepId\n",
      "1\tLuís\tGonçalves\tEmbraer - Empresa Brasileira de Aeronáutica S.A.\tAv. Brigadeiro Faria Lima, 2170\tSão José dos Campos\tSP\tBrazil\t12227-000\t+55 (12) 3923-5555\t+55 (12) 3923-5566\tluisg@embraer.com.br\t3\n",
      "2\tLeonie\tKöhler\tNone\tTheodor-Heuss-Straße 34\tStuttgart\tNone\tGermany\t70174\t+49 0711 2842222\tNone\tleonekohler@surfeu.de\t5\n",
      "3\tFrançois\tTremblay\tNone\t1498 rue Bélanger\tMontréal\tQC\tCanada\tH2G 1A7\t+1 (514) 721-4711\tNone\tftremblay@gmail.com\t3\n",
      "*/\n",
      "\n",
      "\n",
      "CREATE TABLE \"Invoice\" (\n",
      "\t\"InvoiceId\" INTEGER NOT NULL, \n",
      "\t\"CustomerId\" INTEGER NOT NULL, \n",
      "\t\"InvoiceDate\" DATETIME NOT NULL, \n",
      "\t\"BillingAddress\" NVARCHAR(70), \n",
      "\t\"BillingCity\" NVARCHAR(40), \n",
      "\t\"BillingState\" NVARCHAR(40), \n",
      "\t\"BillingCountry\" NVARCHAR(40), \n",
      "\t\"BillingPostalCode\" NVARCHAR(10), \n",
      "\t\"Total\" NUMERIC(10, 2) NOT NULL, \n",
      "\tPRIMARY KEY (\"InvoiceId\"), \n",
      "\tFOREIGN KEY(\"CustomerId\") REFERENCES \"Customer\" (\"CustomerId\")\n",
      ")\n",
      "\n",
      "/*\n",
      "3 rows from Invoice table:\n",
      "InvoiceId\tCustomerId\tInvoiceDate\tBillingAddress\tBillingCity\tBillingState\tBillingCountry\tBillingPostalCode\tTotal\n",
      "1\t2\t2009-01-01 00:00:00\tTheodor-Heuss-Straße 34\tStuttgart\tNone\tGermany\t70174\t1.98\n",
      "2\t4\t2009-01-02 00:00:00\tUllevålsveien 14\tOslo\tNone\tNorway\t0171\t3.96\n",
      "3\t8\t2009-01-03 00:00:00\tGrétrystraat 63\tBrussels\tNone\tBelgium\t1000\t5.94\n",
      "*/\n",
      "==================================================\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "Tool Calls:\n",
      "  model_check_query (call_CY9d6TSLm77f3rg4ecA2JK0f)\n",
      " Call ID: call_CY9d6TSLm77f3rg4ecA2JK0f\n",
      "  Args:\n",
      "    state: {'messages': [{'role': 'human', 'content': \"Which country's customers spent the most in 2009, and how much did they spend?\"}, {'role': 'system', 'content': 'CREATE TABLE \"Customer\" (\\n\\t\"CustomerId\" INTEGER NOT NULL, \\n\\t\"FirstName\" NVARCHAR(40) NOT NULL, \\n\\t\"LastName\" NVARCHAR(20) NOT NULL, \\n\\t\"Company\" NVARCHAR(80), \\n\\t\"Address\" NVARCHAR(70), \\n\\t\"City\" NVARCHAR(40), \\n\\t\"State\" NVARCHAR(40), \\n\\t\"Country\" NVARCHAR(40), \\n\\t\"PostalCode\" NVARCHAR(10), \\n\\t\"Phone\" NVARCHAR(24), \\n\\t\"Fax\" NVARCHAR(24), \\n\\t\"Email\" NVARCHAR(60) NOT NULL, \\n\\t\"SupportRepId\" INTEGER, \\n\\tPRIMARY KEY (\"CustomerId\"), \\n\\tFOREIGN KEY(\"SupportRepId\") REFERENCES \"Employee\" (\"EmployeeId\")\\n}\\n\\nCREATE TABLE \"Invoice\" (\\n\\t\"InvoiceId\" INTEGER NOT NULL, \\n\\t\"CustomerId\" INTEGER NOT NULL, \\n\\t\"InvoiceDate\" DATETIME NOT NULL, \\n\\t\"BillingAddress\" NVARCHAR(70), \\n\\t\"BillingCity\" NVARCHAR(40), \\n\\t\"BillingState\" NVARCHAR(40), \\n\\t\"BillingCountry\" NVARCHAR(40), \\n\\t\"BillingPostalCode\" NVARCHAR(10), \\n\\t\"Total\" NUMERIC(10, 2) NOT NULL, \\n\\tPRIMARY KEY (\"InvoiceId\"), \\n\\tFOREIGN KEY(\"CustomerId\") REFERENCES \"Customer\" (\"CustomerId\")\\n}'}]}\n",
      "    messages: [{'role': 'function', 'content': \"SELECT BillingCountry, SUM(Total) AS TotalSpent\\nFROM Invoice\\nWHERE strftime('%Y', InvoiceDate) = '2009'\\nGROUP BY BillingCountry\\nORDER BY TotalSpent DESC\\nLIMIT 1;\"}]\n",
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mquery_gen\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "Tool Calls:\n",
      "  model_check_query (call_CY9d6TSLm77f3rg4ecA2JK0f)\n",
      " Call ID: call_CY9d6TSLm77f3rg4ecA2JK0f\n",
      "  Args:\n",
      "    state: {'messages': [{'role': 'human', 'content': \"Which country's customers spent the most in 2009, and how much did they spend?\"}, {'role': 'system', 'content': 'CREATE TABLE \"Customer\" (\\n\\t\"CustomerId\" INTEGER NOT NULL, \\n\\t\"FirstName\" NVARCHAR(40) NOT NULL, \\n\\t\"LastName\" NVARCHAR(20) NOT NULL, \\n\\t\"Company\" NVARCHAR(80), \\n\\t\"Address\" NVARCHAR(70), \\n\\t\"City\" NVARCHAR(40), \\n\\t\"State\" NVARCHAR(40), \\n\\t\"Country\" NVARCHAR(40), \\n\\t\"PostalCode\" NVARCHAR(10), \\n\\t\"Phone\" NVARCHAR(24), \\n\\t\"Fax\" NVARCHAR(24), \\n\\t\"Email\" NVARCHAR(60) NOT NULL, \\n\\t\"SupportRepId\" INTEGER, \\n\\tPRIMARY KEY (\"CustomerId\"), \\n\\tFOREIGN KEY(\"SupportRepId\") REFERENCES \"Employee\" (\"EmployeeId\")\\n}\\n\\nCREATE TABLE \"Invoice\" (\\n\\t\"InvoiceId\" INTEGER NOT NULL, \\n\\t\"CustomerId\" INTEGER NOT NULL, \\n\\t\"InvoiceDate\" DATETIME NOT NULL, \\n\\t\"BillingAddress\" NVARCHAR(70), \\n\\t\"BillingCity\" NVARCHAR(40), \\n\\t\"BillingState\" NVARCHAR(40), \\n\\t\"BillingCountry\" NVARCHAR(40), \\n\\t\"BillingPostalCode\" NVARCHAR(10), \\n\\t\"Total\" NUMERIC(10, 2) NOT NULL, \\n\\tPRIMARY KEY (\"InvoiceId\"), \\n\\tFOREIGN KEY(\"CustomerId\") REFERENCES \"Customer\" (\"CustomerId\")\\n}'}]}\n",
      "    messages: [{'role': 'function', 'content': \"SELECT BillingCountry, SUM(Total) AS TotalSpent\\nFROM Invoice\\nWHERE strftime('%Y', InvoiceDate) = '2009'\\nGROUP BY BillingCountry\\nORDER BY TotalSpent DESC\\nLIMIT 1;\"}]\n",
      "=================================\u001b[1m Tool Message \u001b[0m=================================\n",
      "\n",
      "Error: The wrong tool was called: model_check_query. Please fix your mistakes. Remember to only call SubmitFinalAnswer to submit the final answer. Generated queries should be outputted WITHOUT a tool call.\n",
      "==================================================\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "\n",
      "SELECT BillingCountry, SUM(Total) AS TotalSpent  \n",
      "FROM Invoice  \n",
      "WHERE strftime('%Y', InvoiceDate) = '2009'  \n",
      "GROUP BY BillingCountry  \n",
      "ORDER BY TotalSpent DESC  \n",
      "LIMIT 1;\n",
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mquery_gen\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "\n",
      "SELECT BillingCountry, SUM(Total) AS TotalSpent  \n",
      "FROM Invoice  \n",
      "WHERE strftime('%Y', InvoiceDate) = '2009'  \n",
      "GROUP BY BillingCountry  \n",
      "ORDER BY TotalSpent DESC  \n",
      "LIMIT 1;\n",
      "==================================================\n",
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mcorrect_query\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "Tool Calls:\n",
      "  db_query_tool (call_ObYXA2njNIwd5Z7moBPMPh7A)\n",
      " Call ID: call_ObYXA2njNIwd5Z7moBPMPh7A\n",
      "  Args:\n",
      "    query: SELECT BillingCountry, SUM(Total) AS TotalSpent  \n",
      "FROM Invoice  \n",
      "WHERE strftime('%Y', InvoiceDate) = '2009'  \n",
      "GROUP BY BillingCountry  \n",
      "ORDER BY TotalSpent DESC  \n",
      "LIMIT 1;\n",
      "==================================================\n",
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mexecute_query\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "=================================\u001b[1m Tool Message \u001b[0m=================================\n",
      "Name: db_query_tool\n",
      "\n",
      "[('USA', 103.95)]\n",
      "==================================================\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "\n",
      "Answer: The customers from the USA spent the most in 2009, totaling $103.95.\n",
      "\n",
      "==================================================\n",
      "🔄 Node: \u001b[1;36mquery_gen\u001b[0m 🔄\n",
      "- - - - - - - - - - - - - - - - - - - - - - - - - \n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "\n",
      "Answer: The customers from the USA spent the most in 2009, totaling $103.95.\n",
      "==================================================\n"
     ]
    }
   ],
   "source": [
    "output = run_graph(\n",
    "    \"Which country's customers spent the most in 2009, and how much did they spend?\",\n",
    "    stream=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluating an SQL Agent Using LangSmith Evaluator\n",
    "\n",
    "Now, let's evaluate the SQL Agent's responses to queries. To do this, we'll first create a dataset specifically designed for evaluation purposes.\n",
    "\n",
    "Next, we define an evaluator and proceed with the evaluation process.\n",
    "\n",
    "For this, we use **LLM-as-judge** as the evaluator. The prompt employed will be the default one provided by the evaluation prompt hub.\n",
    "\n",
    "However, for more precise evaluations, it is recommended to fine-tune the prompt to suit your specific needs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langsmith import Client\n",
    "\n",
    "# Initialize the client\n",
    "client = Client()\n",
    "\n",
    "# Create and upload the dataset\n",
    "examples = [\n",
    "    (\n",
    "        \"Which country's customers spent the most? And how much did they spend?\",\n",
    "        \"The country whose customers spent the most is the USA, with a total spending of 523.06.\",\n",
    "    ),\n",
    "    (\n",
    "        \"What was the most purchased track of 2013?\",\n",
    "        \"The most purchased track of 2013 was Hot Girl.\",\n",
    "    ),\n",
    "    (\n",
    "        \"How many albums does the artist Led Zeppelin have?\",\n",
    "        \"Led Zeppelin has 14 albums\",\n",
    "    ),\n",
    "    (\n",
    "        \"What is the total price for the album “Big Ones”?\",\n",
    "        \"The total price for the album 'Big Ones' is 14.85\",\n",
    "    ),\n",
    "    (\n",
    "        \"Which sales agent made the most in sales in 2009?\",\n",
    "        \"Steve Johnson made the most sales in 2009\",\n",
    "    ),\n",
    "]\n",
    "\n",
    "dataset_name = \"SQL Agent Response\"\n",
    "\n",
    "# Check if the dataset already exists; if not, create it\n",
    "if not client.has_dataset(dataset_name=dataset_name):\n",
    "    dataset = client.create_dataset(dataset_name=dataset_name)\n",
    "    inputs, outputs = zip(\n",
    "        *[({\"input\": text}, {\"output\": label}) for text, label in examples]\n",
    "    )\n",
    "    client.create_examples(inputs=inputs, outputs=outputs, dataset_id=dataset.id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we define a function to predict the SQL query response generated by our agent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a function to predict the SQL query response from the agent\n",
    "def predict_sql_agent_answer(example: dict):\n",
    "    \"\"\"Use this for answer evaluation\"\"\"\n",
    "    config = RunnableConfig(\n",
    "        configurable={\"thread_id\": random_uuid()},\n",
    "        recursion_limit=5,  # If the evaluation takes a long time, adjust the recursion limit appropriately.\n",
    "    )\n",
    "\n",
    "    inputs = {\n",
    "        \"messages\": [HumanMessage(content=example[\"input\"])],\n",
    "    }\n",
    "    # Execute the graph and retrieve the message result\n",
    "    try:\n",
    "        messages = app.invoke(inputs, config)\n",
    "        answer = messages[\"messages\"][-1].content\n",
    "        # Return the result\n",
    "        return {\"response\": answer}\n",
    "    except:\n",
    "        return {\"response\": \"Error: The agent failed to generate a response.\"}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define the prompt and the evaluator (LLM-as-judge) to evaluate the SQL query response."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain import hub\n",
    "from langchain_openai import ChatOpenAI\n",
    "\n",
    "# Grade prompt for answer accuracy\n",
    "grade_prompt_answer_accuracy = hub.pull(\"langchain-ai/rag-answer-vs-reference\")\n",
    "\n",
    "\n",
    "# Define the answer evaluator (LLM-as-judge)\n",
    "def answer_evaluator(run, example) -> dict:\n",
    "    # Input: the question\n",
    "    input_question = example.inputs[\"input\"]\n",
    "    # Output: reference answer\n",
    "    reference = example.outputs[\"output\"]\n",
    "    # Predicted answer\n",
    "    prediction = run.outputs[\"response\"]\n",
    "\n",
    "    # Initialize the LLM evaluator\n",
    "    llm = ChatOpenAI(model=MODEL_NAME, temperature=0)\n",
    "    answer_grader = grade_prompt_answer_accuracy | llm\n",
    "\n",
    "    # Execute the evaluator\n",
    "    score = answer_grader.invoke(\n",
    "        {\n",
    "            \"question\": input_question,\n",
    "            \"correct_answer\": reference,\n",
    "            \"student_answer\": prediction,\n",
    "        }\n",
    "    )\n",
    "    score = score[\"Score\"]\n",
    "\n",
    "    # Return the score\n",
    "    return {\"key\": \"answer_v_reference_score\", \"score\": score}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we perform the evaluation and review the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "View the evaluation results for experiment: 'sql-agent-eval-49288735' at:\n",
      "https://smith.langchain.com/o/78e3709b-1bda-5e4a-b235-fb646a97dd19/datasets/4697171d-750c-4d54-a7ba-ea314abd4c46/compare?selectedSessions=53339ff5-44e6-44c4-bf91-b1a58f149953\n",
      "\n",
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c7d049bf7ad64e989dc4e1ded325f15d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "Tool Calls:\n",
      "  model_check_query (call_60DijUCPZ6b5Lfh9nlr2Ptto)\n",
      " Call ID: call_60DijUCPZ6b5Lfh9nlr2Ptto\n",
      "  Args:\n",
      "    state: {'messages': [{'role': 'human', 'content': 'Which sales agent made the most in sales in 2009?'}, {'role': 'system', 'content': 'The Employee table contains information about employees, including their titles. The Invoice table contains sales data, including the total amount for each invoice. The InvoiceLine table contains details about each item sold in an invoice.'}, {'role': 'function', 'content': \"SELECT e.FirstName, e.LastName, SUM(i.Total) AS TotalSales\\nFROM Employee e\\nJOIN Invoice i ON e.EmployeeId = i.CustomerId\\nWHERE strftime('%Y', i.InvoiceDate) = '2009'\\nGROUP BY e.EmployeeId\\nORDER BY TotalSales DESC\\nLIMIT 1;\"}]}\n",
      "    messages: [{'role': 'function', 'content': \"SELECT e.FirstName, e.LastName, SUM(i.Total) AS TotalSales\\nFROM Employee e\\nJOIN Invoice i ON e.EmployeeId = i.CustomerId\\nWHERE strftime('%Y', i.InvoiceDate) = '2009'\\nGROUP BY e.EmployeeId\\nORDER BY TotalSales DESC\\nLIMIT 1;\"}]\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "Tool Calls:\n",
      "  model_check_query (call_OugYsX1NOWkM0qd7YF7cXStc)\n",
      " Call ID: call_OugYsX1NOWkM0qd7YF7cXStc\n",
      "  Args:\n",
      "    state: {'messages': [{'content': \"SELECT SUM(InvoiceLine.UnitPrice * InvoiceLine.Quantity) AS TotalPrice\\nFROM InvoiceLine\\nJOIN Track ON InvoiceLine.TrackId = Track.TrackId\\nJOIN Album ON Track.AlbumId = Album.AlbumId\\nWHERE Album.Title = 'Big Ones';\", 'type': 'ai'}, {'content': \"What is the total price for the album 'Big Ones'?\", 'type': 'human'}]}\n",
      "    usage_metadata: {'input_tokens': 36, 'output_tokens': 0, 'total_tokens': 36}\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "Tool Calls:\n",
      "  model_check_query (call_HCkCDz711l7pmtjEXgNU2gkc)\n",
      " Call ID: call_HCkCDz711l7pmtjEXgNU2gkc\n",
      "  Args:\n",
      "    state: {'messages': [{'content': 'How many albums does the artist Led Zeppelin have?', 'type': 'human'}, {'content': 'CREATE TABLE \"Album\" (\\n\\t\"AlbumId\" INTEGER NOT NULL, \\n\\t\"Title\" NVARCHAR(160) NOT NULL, \\n\\t\"ArtistId\" INTEGER NOT NULL, \\n\\tPRIMARY KEY (\"AlbumId\"), \\n\\tFOREIGN KEY(\"ArtistId\") REFERENCES \"Artist\" (\"ArtistId\")\\n)\\n\\n/*\\n3 rows from Album table:\\nAlbumId\\tTitle\\tArtistId\\n1\\tFor Those About To Rock We Salute You\\t1\\n2\\tBalls to the Wall\\t2\\n3\\tRestless and Wild\\t2\\n*/\\n\\nCREATE TABLE \"Artist\" (\\n\\t\"ArtistId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"ArtistId\")\\n)\\n\\n/*\\n3 rows from Artist table:\\nArtistId\\tName\\n1\\tAC/DC\\n2\\tAccept\\n3\\tAerosmith\\n*/', 'type': 'system'}]}\n",
      "    tool_calls: [{'name': 'functions.model_check_query', 'args': {'state': {'messages': [{'content': 'How many albums does the artist Led Zeppelin have?', 'type': 'human'}, {'content': 'CREATE TABLE \"Album\" (\\n\\t\"AlbumId\" INTEGER NOT NULL, \\n\\t\"Title\" NVARCHAR(160) NOT NULL, \\n\\t\"ArtistId\" INTEGER NOT NULL, \\n\\tPRIMARY KEY (\"AlbumId\"), \\n\\tFOREIGN KEY(\"ArtistId\") REFERENCES \"Artist\" (\"ArtistId\")\\n)\\n\\n/*\\n3 rows from Album table:\\nAlbumId\\tTitle\\tArtistId\\n1\\tFor Those About To Rock We Salute You\\t1\\n2\\tBalls to the Wall\\t2\\n3\\tRestless and Wild\\t2\\n*/\\n\\nCREATE TABLE \"Artist\" (\\n\\t\"ArtistId\" INTEGER NOT NULL, \\n\\t\"Name\" NVARCHAR(120), \\n\\tPRIMARY KEY (\"ArtistId\")\\n)\\n\\n/*\\n3 rows from Artist table:\\nArtistId\\tName\\n1\\tAC/DC\\n2\\tAccept\\n3\\tAerosmith\\n*/', 'type': 'system'}]}, 'tool_calls': []}}]\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "Tool Calls:\n",
      "  model_check_query (call_kXTpc9bApkYr0bOmdUaGUkNq)\n",
      " Call ID: call_kXTpc9bApkYr0bOmdUaGUkNq\n",
      "  Args:\n",
      "    state: {'messages': [{'content': \"SELECT Track.Name, COUNT(InvoiceLine.TrackId) AS PurchaseCount\\nFROM InvoiceLine\\nJOIN Track ON InvoiceLine.TrackId = Track.TrackId\\nJOIN Invoice ON InvoiceLine.InvoiceId = Invoice.InvoiceId\\nWHERE strftime('%Y', Invoice.InvoiceDate) = '2013'\\nGROUP BY Track.TrackId\\nORDER BY PurchaseCount DESC\\nLIMIT 1;\", 'type': 'ai'}]}\n",
      "==================================\u001b[1m Ai Message \u001b[0m==================================\n",
      "Tool Calls:\n",
      "  model_check_query (call_TzwL1JFwP71RLYk9u08jLPa9)\n",
      " Call ID: call_TzwL1JFwP71RLYk9u08jLPa9\n",
      "  Args:\n",
      "    state: {'messages': [{'content': 'SELECT Country, SUM(Total) AS TotalSpent \\nFROM Invoice \\nJOIN Customer ON Invoice.CustomerId = Customer.CustomerId \\nGROUP BY Country \\nORDER BY TotalSpent DESC \\nLIMIT 1;', 'type': 'ai'}]}\n"
     ]
    }
   ],
   "source": [
    "from langsmith.evaluation import evaluate\n",
    "\n",
    "# Name of the evaluation dataset\n",
    "dataset_name = \"SQL Agent Response\"\n",
    "\n",
    "try:\n",
    "    # Perform the evaluation\n",
    "    experiment_results = evaluate(\n",
    "        predict_sql_agent_answer,  # Prediction function used for evaluation\n",
    "        data=dataset_name,  # Name of the evaluation dataset\n",
    "        evaluators=[answer_evaluator],  # List of evaluators\n",
    "        num_repetitions=1,  # Number of experiment repetitions\n",
    "        experiment_prefix=\"sql-agent-eval\",\n",
    "        metadata={\n",
    "            \"version\": f\"chinook db, sql-agent-eval: {MODEL_NAME}\"\n",
    "        },  # Experiment metadata\n",
    "    )\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The evaluation results can be reviewed individually at the generated URL.\n",
    "\n",
    "![evaluation-results](./assets/09-sql-agent-evaluation-results.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "langchain-opentutorial-BpEVOGYk-py3.12",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
